Achieving personalized outcomes with digital therapeutic applications

ABSTRACT

Systems, methods, and devices, including computer-readable media, for managing operation of devices in complex systems and changing environments. In some implementations, a server system stores data indicating management plans for each of a plurality of different devices, each management plan indicating a device-specific set of program states for programs in a predetermined set of programs. The server system alters the management plans and enforces interdependence of the programs, and the server system generates a customized instruction that alters operation of the device according to the device-specific set of program states assigned in the altered management plan for the device. The server system causes each device to perform one or more operations of the device determined according to the device-specific set of program states assigned in the altered management plan for the device.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of U.S. Ser. No. 15/803,556,filed on Nov. 3, 2017, the contents of which are incorporated byreference herein.

TECHNICAL FIELD

This document generally relates to computing techniques that can be usedto efficiently manage networked devices, including making fine-grainedcontext-dependent adjustments to the operation of devices in complexand/or distributed systems.

BACKGROUND

Management of devices in complex systems can be challenging. In manycases, equipment is managed using manual intervention anddeterminations, for example, repairing devices when components fail.Although many devices of similar capabilities may be employed, thephysical state of each device may be unique (e.g., maintenance status,component status, battery life, etc.) and the context and tasks of eachdevice may also be unique and variable. Yet devices of a similar typeare often controlled in the same manner regardless of varying physicalstate and context, often with detrimental long-term results. When aportion of a system malfunctions or needs repair, a significant sectionor even the entire system may be affected. For example, in amanufacturing environment, a first machine's failure or poor qualityoutput may result in waste or in idling of many other machines thatrequire the first machine's output to perform their tasks properly.

SUMMARY

In some implementations, a system creates and manages dynamic,customized plans for managing networked devices in complex systems,including networked devices that may or may not interact directly withusers. The system can coordinate operating programs that affectdifferent operational characteristics, including changes to alter wear,output, interactions, risks, and user-facing behavior. The dynamicoperating programs allow the system to interact with various othersystems to automatically adjust the content of each management plan. Thesystem also carries out the plans by issuing real-time commands. Thesystem can react to each device's current situation with contextuallyrelevant changes to the device's management plan. The interactions ofthe system are not limited to the device. The system can also interactwith other devices that are connected to (e.g., in communication with)or related to the device, such as devices of plant managers, maintenanceworkers, or other users. The system can also assess unique risk levelsfor each device and predictively act to mitigate those risks. Further,the system can use machine learning techniques to identify thecombinations of interventions that best improve outcomes for differentdevices or groups of devices.

One of the advantages of the system is the ability to use a common setof digital operating programs to provide unique, adaptive, customizedinstructions for a wide variety of complex systems. The system may alsoprovide real-time or near-real-time responsiveness to adjust for adevice's current status and situation.

In general, maintaining and updating sophisticated software can bedifficult, especially as computer systems and requirements change overtime. A particular challenge is managing the dependency among differentelements of a system. Traditional systems often require staticdependency among software components or impose other constraints onrelationships between modules, and changes to one component oftenrequire updates to many other related components to accommodate thechanges. Some systems may use independent components, but withsignificantly less flexibility and greater overhead, since thecomponents may not effectively share information.

The present application describes solutions to the inflexibility andinefficiency of many software platforms with a new technique to addressdependency among sophisticated software systems and vary managementplans with low overhead. In some implementations, the system uses acollection of modules or programs hosted by a server system. The sameset of modules or programs are used together for the creation, carryingout, and ongoing adaptation of management plans for many differentdevices, yet the content of the programs and interrelationships allowfor vast numbers of unique management plans through differentcombinations and settings for the programs. The programs can operate inparallel and asynchronously to each other, and can share the same datasets to reduce storage requirements and overhead. As discussed furtherbelow, each program may be capable of operating at any of multipledifferent states, where each program state corresponds to differentbehavior of the program, e.g., using different sets of rules for deviceactions and different sets of interactive content. The set of programscan, collectively and in different combinations, address the uniqueneeds and circumstances of many different devices, with the system usinga device-specific or user-specific set of program states for each devicethat it manages. For example, if a system has three programs, each withstates 1-10, a first device may be supported with the programs at states1, 3, and 2, a second device may be supported by the same programsoperating at states 2, 3, and 5, and so on. The programs can also usecommon data sets (e.g., rule sets, content repositories, controlparameters such as thresholds, etc.) for each device, which areavailable to all programs, to minimize storage requirements and latency.

The programs can be designed to operate in any of the given programstates without dependency on any of the other programs, so thatoperation or delay of any program does not block operation of otherprograms. Nevertheless, the system enables interdependence among theprograms in the manner that the program states are set. For example, thesystem can dynamically set the program states of programs for a deviceaccording to the program states of other programs for the same device.For example, although each program can represent a self-contained unitthat can operate and interact with devices separate from other programs,the system may also take into account a device's user-specific programstates for all programs when the system evaluates whether to dynamicallychange program states. Based on the state of one program, the system maydetermine to change the state of other programs, to achieve anappropriate combination of programs active at any given time for aparticular user. The changes in program state can be made periodicallyand asynchronously, so that other programs can continue in their currentstates or change states as needed.

In addition, by sharing the same programs and discrete states of thoseprograms for many devices, the system can provide many differentcombinations of experiences with a small number of programs and states.This maximizes the ability of the system to customize interaction andadapt for individual devices or users, while minimizing storagerequirements and effort to maintain the system. For example, a systemthat uses ten programs with ten states each has a total of 100individual program states, but provides 10¹⁰ or 10 billion differentcombinations of program states, where each combination can provide adifferent management plan. This allows a very large level of flexibilityand customizability with high efficiency, e.g., small storage space andminimal computation for the scope of variability allowed. Further, foreach combination of program states, the operation of the programs (e.g.,application of the subset of rules corresponding to the applicableprogram states) further customizes interaction with the device accordingto a current context, history of interactions, and other data specificto a particular device or its user.

In some implementations, the system includes multiple digital operatingprograms that each relate to a different aspect of the device'soperation or a user's needs. Each program can have a corresponding setof rules that specify system actions to be performed when appropriatetriggers and conditions are satisfied. The system can also use theoutput of these programs and the state of the programs to adjust thenature of the user's management plan. The various programs can beinterdependent, with different programs having states that are modifiedby the system based at least in part on the state of other programs. Inthis manner, programs that are not directly related in topic or functionto other programs can nevertheless affect each other's state (e.g.,whether the program is activated, and at what level or intensity). Themanner in which these programs interact (e.g., how programs andconditions trigger adjustments in other programs) can be learned by thesystem through machine learning techniques.

The device management technique described in the present specificationprovides various advantages over existing management techniques. Forexample, the present method allows individualized management for eachdevice, which can have unique operating characteristics, which can avoidor mitigate the need for maintenance by proactively and dynamicallyadjusting operation according to the devices unique context and state.

Because the operating characteristics of a device may be interrelated ormay combine during certain tasks or activities, and a change to oneoperating characteristic of a device can affect or induce a change in adifferent operating characteristic, the device management plan canaccount for changes to a particular operating characteristic by changinglevels of a particular program in response to changes to an operatingcharacteristic.

The generation of management plans can be performed by a user,automatically, or as a combination (e.g., automatically with input froma user). For example, a management plan can be generated using a set ofprogrammed guidelines, with certain variables that can be input by auser. In some examples, the management plans can be generatedautomatically through the application of machine learning techniques.For example, operational data of devices in a similar context orsituation can be collected over time, and trends and patterns observedcan be used as training examples use to determine the training state ofa classifier, neural network, or other machine learning model, which canlearn to indicate combinations of program states that are appropriatefor different situations.

The management method can be applied to other complex systems, such asnetwork-connected devices in distributed environments. Some figures anddescription below refer to devices in a factory for ease and simplicityof explanation. The same techniques are also applicable in a widevariety of other settings, such as telecommunications, managing robotsor autonomous vehicles, providing digital therapeutics or precisionmedicine, and so on.

In one general aspect, a system includes one or more computers of aserver system, the one or more computers being configured to carry outand update management plans that adjust operation of one or moredevices, where the management plans involve different combinations ofprograms in a predetermined plurality of programs, each of the programshaving multiple predetermined program states specifying different levelsof interaction or different levels of intensity of the correspondingprogram. The server system includes a data storage subsystem configuredto store data indicating a management plan for a device, the managementplan indicating a device-specific set of program states for theplurality of programs. The server system includes a program statesetting module configured to alter the management plan for the device byassigning, in the altered management plan, an altered device-specificset of program states corresponding to the plurality of programs, wherethe program state setting module enforces interdependence among theprograms such that the program states for the device for one or more ofthe programs are dependent on the program states for the device for oneor more of the other programs. The server system includes an instructiongeneration module configured to generate, for the device, a customizedinstruction that alters operation of the device according to thedevice-specific set of program states assigned in the altered managementplan for the device, where the instruction generation module determinesthe customized instruction by evaluating, for each of the programs thatis active according to the altered management plan for the device, asubset of rules for the program, the subset being associated with aparticular program state for the program indicated in the alteredmanagement plan. The server system includes a transmission moduleconfigured to send, to the device over a communication network, theinstruction generated for the device by the instruction module accordingto the device-specific set of program states assigned in the alteredmanagement plan for the device.

In some implementations, the one or more computers are configured tocarry out and update device-specific management plans for a plurality ofdifferent devices, each of the management plans using differentcombinations of programs in the predetermined set of programs. Theserver system is configured to store data indicating the managementplans and device-specific program states for each of the plurality ofdifferent devices. The program state setting module is configured toalter the management plans for each of the plurality of differentdevices. The instruction generation module is configured to generate acustomized instruction for each of the plurality of devices according tothe different sets of device-specific program states for the pluralityof devices. The transmission module is configured to send the respectivecustomized instructions to the plurality of devices over thecommunication network.

In some implementations, the program state setting module is a machinelearning model. For example, the program state setting module caninclude a neural network, a regression model, a decision tree, a supportvector machine, a maximum entropy classifier, or other machine learningmodel.

In some implementations, the server system is configured to communicateindividualized application content to the device over the communicationnetwork. The server system generates the individualized applicationcontent using the plurality of programs and device-specific sets ofprogram states of the programs for the device.

In some implementations, the instruction is configured to cause thedevice to record a measurement from one or more devices that are incommunication with the device through a local wired or wireless dataconnection.

In some implementations, the server system is configured to: (1) storeprogram status data indicating a set of multiple of the programs thatare active for the device a particular time, the program status dataalso indicating a program state for each of the multiple programs thatare active at the particular time, the program state for each programbeing selected from among the predetermined program states of theprogram; (2) define a digital state marker as a particular combinationof programs being concurrently active with particular program states setfor the active programs; (3) identify a group of devices that havepreviously exhibited the digital state marker by having the particularcombination of programs being concurrently active with the particularprogram states set for the active programs; and (4) determine subsequentprogram state transitions that were made by the devices in the groupafter exhibiting the digital state marker.

In some implementations, the server system is configured to: (1)determine that the device exhibits the digital state marker based on thedevice-specific program states for the device indicating that theparticular combination of programs is currently active with theparticular program states set for the active programs; (2) automaticallyvary the device-specific program states of the plurality of programs toassign updated device-specific program states for the device, where theserver system assigns the updated device-specific program states for thedevice in response to determining that the device exhibits the digitalstate marker and based on the program state transitions made for thedevices in the group after the devices in the group exhibited thedigital state marker; (3) generate individualized application contentfor the device by applying, for each of the active programs, the subsetof rules associated with the updated program state for the program; and(4) distribute the individualized application content to the device, theindividualized application content being generated for the deviceaccording to the updated device-specific program states for the device.

In some implementations, the instruction is configured to cause thedevice to provide an output at the device, where the output includespresentation of text, image, audio, or video by the device or anotherdevice in communication with the device.

In some implementations, to alter the management plan for the device,the server system is configured to: automatically enable one or moredigital therapeutics programs for a particular user such that the serversystem initiates use of the enabled program to generate individualizedapplication content, communicate the individualized application contentto the device, and instruct the device to present the individualizedapplication content; or automatically disable one or more digitaltherapeutics programs for the particular user such that the serversystem discontinues use of the disabled program to generateindividualized application content for the particular user.

In some implementations, one or more of the program states of a programcorrespond to different time periods representing sequential progressionthrough a set of predetermined elements of the program.

In some implementations, one or more of the program states of a programcorrespond to different levels of intensity or frequency for monitoring,adjusting, or instructing actions of the device, or correspond todifferent levels of interaction with a user of the device.

In some implementations, the device is a first device, and where theserver system is configured to: identify an attribute to be determinedfor the first device; provide, to a second device, an instruction toacquire information indicative of the attribute of the first device or auser of the first device; and provide, to the first device, one or moreinstructions to adjust operation of the first device based oninformation indicative of the attribute of the first device or user ofthe first device that was received from the second device in response tothe instruction to the second device.

In some implementations, the server system is configured to: obtainsensor data from the device; and vary one or more program states of themanagement plan for the device automatically based at least in part onthe sensor data generated by the device.

In some implementations, the server system is configured to: vary theprogram states of a management plan for the device, the device beingassociated with a user, where the program states are varied by theserver system according to data from (i) at least one general datasource providing one or more of weather data, points of interest data,or environmental data, and (ii) at least one user-specific data sourcefrom the group consisting of data from a wireless wearable device of theuser, data from a medical device of the user, electronic health recordsfor the user, prescription data for the user, genetic data for the user,social media data for the user, and mobile device data from a mobiledevice associated with the user.

In some implementations, the server system is configured toautomatically vary the program states each of a plurality of differentdevices based on behavioral patterns of respective users of theplurality of different devices to instruct the plurality of differentdevices to initiate digital therapeutics interventions for therespective users of the plurality of different devices.

In some implementations, the server system is configured to provide datafor an administrator interface indicating the device-specific programstates for each of one or more of the different devices; where theadministrator interface includes controls enabling the administrator tochange program states that were automatically assigned by the serversystem.

In some implementations, the server system is configured to filter rulesof the programs to determine the subsets to evaluate. The rules can bepre-associated with different programs and program states, and theserver system can filter the rules to select a subset of rules for eachprogram, in which each of the rules in the subset for a program isassociated with the program state indicated for the program by themanagement plan for the device.

In some implementations, the server system is configured to transmit thegenerated instruction such that the instruction causes the device tocarry out one or more actions that alter operation of the device.

In another general aspect, a method includes: storing, by a serversystem, data indicating a management plan for a device, the managementplan indicating a device-specific set of program states for programs ina predetermined plurality of programs, each of the programs havingmultiple predetermined program states specifying different levels ofinteraction or different levels of intensity of the correspondingprogram; altering, by the server system, the management plan for thedevice by assigning, in the altered management plan, an altereddevice-specific set of program states corresponding to the plurality ofprograms, where the server system enforces interdependence of theprograms such that the program states for the device for one or more ofthe programs are dependent on the program states for the device for oneor more of the other programs; generating, by the server system, aninstruction that alters operation of the device according to thedevice-specific set of program states assigned in the altered managementplan for the device, where the instruction is determined by evaluating,for each of the programs that is active according to the alteredmanagement plan, a proper subset of rules for the program, the subsetbeing associated with a particular program state for the programindicated in the altered management plan; and transmitting, by theserver system and to the device over a communication network, thegenerated instruction for the device determined according to thedevice-specific set of program states assigned in the altered managementplan for the device.

Other embodiments of these aspects include corresponding systems,apparatus, and computer programs, configured to perform the actions ofthe methods, encoded on computer storage devices. A system of one ormore computers can be so configured by virtue of software, firmware,hardware, or a combination of them installed on the system that inoperation cause the system to perform the actions. One or more computerprograms can be so configured by virtue of having instructions that,when executed by data processing apparatus, cause the apparatus toperform the actions.

The details of one or more embodiments of the subject matter describedin this specification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages of thesubject matter will become apparent from the description, the drawings,and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram that illustrates an example system for managingdevices.

FIG. 2 is a diagram that illustrates an example learning and trainingsystem for managing devices.

FIG. 3A is a diagram that illustrates an example of a generalized systemfor generating dynamic, customized management plans for each of thedevices.

FIGS. 3B, 3C, 3D, and 3E illustrate examples of user interfaces.

FIG. 4 shows an example of a system for generating an operating plan.

FIG. 5 is a diagram that illustrates examples of data used to generatedynamic, customized operating plans.

FIGS. 6A and 6B are diagrams that illustrate examples of techniques foradjusting states of operating plan programs for devices in complexsystems.

FIG. 7 is a diagram that illustrates examples of using activityinformation to develop an operating plan for devices in complex systems.

FIG. 8 is a diagram that illustrates examples of data that can be usedby various operating plan programs for devices in complex systems.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

FIG. 1 is a diagram that illustrates an example of a system for managingdevices of a factory 101 that includes multiple different devices 120 a,120 b, and 120 c (collectively, 120). Each of devices 120 a, 120 b, and120 c is different, and has different characteristics and differentoperating parameters. The devices 120 can be, for example, devices infactor 101. Factory 101 can includes devices of the same type, e.g.,some devices can be the same models and from the same manufacturer;while the devices are of the same model and from the same manufacturer,each of the devices can be unique, and can have unique characteristicsthat affect the operation and efficiency of each device. For example, adevice may be 4 years old, and may have a timing belt of a differentthickness from other devices of its make and model. In another example,a different device of the same make and model may be 2 weeks old, andits components may require a break-in period. Thus, while devices canhave the same general capabilities or roles, the devices can havedifferent optimal operating conditions and create products of differentqualities at different rates.

The system 100 generates dynamic, customized operating plans ormanagement plans for each of the devices. The system 100 includes aserver system 110 and an administrator system 130. The server system 110communicates with each of the other components of the system over anetwork 116. The server system 110 also accesses a number of datacollections, illustrated as databases 111, program rules 112, andprogram content 113.

The example in FIG. 1 shows various interactions that can be used tocreate, adjust, and carry out management plans. As described above, thesystem 100 can generate management plans for a variety of devices. Eachdevice may be associated with one or more users. The characteristics orbehavior of the user can be taken into account in determining theprograms and levels that are active for a given device. Thus, amanagement plan may guide or assist in managing actions of the user ofthe device as well, not only the direct operation of the device, throughthe outputs and interactions of the device as the management plan isapplied.

As an example, interactions shown in stages (A) to (E) follow a processof creating management plans for devices, monitoring each device'sstatus and needs, providing instructions and commands, and adjusting themanagement plans for each device. The management plan provided by thesystem is not merely a statement of desired device performance oroperating plans. Rather, the management plan provided by the system 100is a real-time, interactive customized service that initiatescommunication and interaction with the device or user to carry outoperating plans that adjust how devices operate. Among other effects,the management plans can involve interactions, initiated by the system110 or by a user, that change the device's performance or operation inone or more aspects. The system 100 can operate in an “always-on”manner, frequently or continually assessing a data stream indicating thecurrent status and context of a device (and thus the device's useralso), and providing targeted interactions that are relevant to thecurrent and/or estimated future needs of the device and thecorresponding user.

During stage (A), the server system 110 generates and provides anadministrator portal 132, which may be provided as a web page, webapplication, data for a locally executable application of theadministrator system 130, or in another form. This interactive portal132 enables the administrator to view management plan templates 134 andselect a template to use to enroll a new device, such as a machine orpiece of equipment. The portal 132 also provides controls for theadministrator to modify the templates 134 for individual devices, aswell as view and alter management plans that are currently active forany of the devices assisted by the administrator.

The administrator portal 132 provides a number of management plantemplates 134 that the administrator can select when enrolling a newdevice. In some instances, the templates correspond to different areasor fields that a user wishes to manage. The management plans can begenerated as combinations of different digital operating programs. Atemplate 134 can specify a set of these programs that are made inactivefor a given device. The programs each have a number of components andstates that very how they interact with the device. For example, eachprogram may include multiple segments that are applicable over differenttime periods. The program can define a sequence of these segments, ormultiple different sequences. The segments may correspond to fixedlength periods, for example, a week, or month, or another fixed orstandardized time period. Alternatively, one or more of the periods mayhave a variable duration.

Each program has various levels or intensities with which the programapplies. Different levels may correspond to different levels of need. Insome implementations, different levels of a program, may respond todifferent objectives or conditions.

When a device is first enrolled, the management plan template 134selected by the administrator specify a default set of programs andcorresponding levels for the device. The administrator may manuallyadjust the selections, for example, by adding or removing programs, andincreasing or decreasing the levels of programs. In this matter, theadministrator can specify an initial management plan that meets theneeds identified by the device.

With the interfaces and tools provided by the server system 110 throughthe administrator portal 132, an administrator can efficiently managemanagement plans for a large number of devices. In many instances, afterthe management plan is initiated, the administrator does not need tofurther modify or adjust the management plan. The server system 110automatically adjusts each individual management plan according to theunique set of user input and sensor data detected for each device.Nevertheless, an administrator can view an individual device's progressand, if appropriate, adjust a device's management plan manually throughthe administrator interface 132. As the server system 110 adjusts themanagement plans based on each device's individual circumstances,different management plans will have different segments of programs,different levels of the programs, and different combinations of theprograms active. These differences as well as the application ofindividual program rules to unique device data sets, provide a uniqueexperience to each user.

During stage (A), the server system 110 stores information about eachdevice enrolled in the system. The server system 110 can store theinformation using a data storage subsystem which may include, forexample, direct attached storage (e.g., hard drives, solid state drives,etc.), locally attached storage, network attached storage (NAS), one ormore databases, a storage area network (SAN), a database managementsystem (DBMS) and/or one or more database servers, or other elements.For example, the data storage subsystem may include one or morerelational databases (e.g., an SQL database), one or moreobject-oriented databases, one or more noSQL databases (e.g., akey-value database, a wide column store database, etc.), a distributeddata store, one or more file systems, or other data storage system. Thisdata storage subsystem can store data gathered from devices beingmanaged (e.g., state information, sensor data, operation history, etc.)as well as information that the server system 110 generates (e.g.,current and prior levels for different programs, instructions todevices, source content and generated content to be transmitted todevices, etc.). As discussed further below, the server system 110 storesdata indicating the current state of a device's management plan, such asthe programs that are active, the levels of each program, which segmentthe user is currently in for each program, and so on. In addition, theserver system 110 stores historical information about the device andits/his interactions with the application and progress while using themanagement plan. The server system 110 can acquire data from manydifferent sources including surveys, sensor data, and other datadiscussed further below. This information can be stored in variousdatabases 111.

The operating characteristics of a device can be determined and used toadjust the operation of a device to optimize for different goals. Adevice's operation can be adjusted for improved efficiency, maximumaccuracy, increased speed, etc. By performing actions to adjust theoperating characteristics different goals can be achieved. These actionscan be determined based on the operating characteristics, and can begoverned by a plan for each device 102. The action can be selected basedon factors including underlying causes for the device's operatingcharacteristics. Each action can affect or account for operatingconditions of the devices. For example, if a device has low accuracy dueto operating at a high spindle speed, actions can include slowing downthe production rate of the device (e.g., so the spindle speed decreasesand accuracy increases), adding quality control checks to the device'soutputs, etc. The operation of other devices may also be adjusted inresponse, for example, so that a second device implements more frequentor detailed quality checks for the first device, or so that anotherdevice increases production rate to compensate for the decreasedproduction of the first device.

As another example, if a device consistently needs to be shut down for acooling-down period, slowing down the production rate of the device candecrease the frequency with which the device needs to cool-down. Forexample, a high work pace can induce high heat levels at various pointsand components of the device. A reduction in pace affects variousoperating characteristics of the device, such as tendency to overheat,work piece tolerances, etc.

The actions can depend on underlying and/or interrelated characteristicsof the device. A change in the spindle speed of the device can produceeffects in addition to an intended or expected effect. For example, if adevice's work pace is faster than other devices, the rushed productioncycle can reduce the accuracy of a device's work, and the fast pace canincrease the heat levels at the device at various moving parts, thusincreasing the frequency with which the device must be cooled down. Inanother example, flagging the device's work for review decreases thespeed at which the device can turn out products, and increases accuracywhile decreasing the chances of overheating the device. With feweroverheating incidents, the frequency of unexpected repairs to the devicecan decrease.

During stage (B), the server system 110 also stores information thatdefines each of the programs. Each program can include a correspondingset of program rules 112. A repository of these rules 112 is maintainedand accessed by the server system 110, and the rules are evaluated on anongoing basis to determine when to communicate with each device and inwhat manner. If a device has multiple programs active, rules for eachprogram separately can specify different communications to be providedto the device.

In some applications, devices can have common stages within theiroperational lifetime, or can have common operating conditions for whicha general set of actions can account. For example, if devices nearingthe end of their useful life commonly have lower accuracy and lowerefficiency, and these degradations in work product quality can beaddressed using a common set of actions across different devices ordifferent types of devices, the common set of actions can be collectedand applied to each device at the end stage of its operational lifetime.This common set of actions can be standardized as a program, and can begenerated based on data collected from multiple devices over time. Theseprograms can be tailored to different aspects of the devices. Forexample, a program for older devices can include preventative measuresto decrease the chances of unexpected failures by scheduling morefrequent quality assessments, lower production targets and speeds,flagging the device for review, etc. If older devices are scheduled formore frequent maintenance checks, any potential problems can bediagnosed and repaired as soon as they present.

The rules for the programs have conditions and triggers associated withthem. Each rule may have a condition that must be satisfied for acorresponding system action of the rule to be performed. In addition, arule can have a trigger or triggering condition associated with it, sothat the action of the rule occurs when the trigger is satisfied as wellas one or more other associated conditions.

The actions of the server system 110 to generate user specificinterfaces and content can draw from a repository of program content113. This repository 113 can include media, messages, and other contentthat is selectively provided according to the program rules in therepository 112, which in turn are selectively used according to whichprograms are active, which levels of the programs are active, and whichsegments of the programs are active.

The server system 110 provides a set of program data 150 that indicatesthe active programs and levels of each active program for each of thedevices 120 a, 120 b, and 120 c.

During stage (C), the server system 110 obtains data for each of thevarious users enrolled with the management plan. The server system 110communicates with devices 120 to acquire data for each of the variousdevices in rolled with the management plan. The server system 110communicates with devices 120 to acquire context information, such asthe location of the respective device, a current activity of the device,sensor data, and other data. The server system 110 also communicateswith third-party systems to access data such as device activity andinteractions, and even general information such as current news,weather, and other data.

In stage (D), the server system 110 applies the individual managementplan for each device to the data collected for the device. For example,the rules that are applicable to an active segment of an active programare evaluated using the context information historical information andother data. Although different programs may indicate different messagesto be provided or different instructions to present to a device, theserver system 110 may integrate the content across the various programs.

During stage (E), the server system 110 adjusts the states of theprograms if appropriate. The server system 110 can assess variousfactors to determine whether the set of programs that is active shouldchange, and whether the levels and active segments of each programshould change. In some instances, a device progresses through segmentsof a program based on the passage of time. The server system 110 candetect when an appropriate transition from one time. To the next occurs,and so when the change in applicable rules occurs.

The server system 110 can include a program state setting module toevaluate and change the states of programs in each management plan.Using the program state setting module, the server system 110 can assessvarious factors to determine whether the set of programs that is activeshould change, and whether the levels and active segments of eachprogram should change. In some instances, a device or user progressesthrough segments of a program based on the passage of time. The programstate setting module of the server system 110 can detect when anappropriate transition should occur from one time period or one sectionof a program to another. In general, the program state setting modulecan include software and/or hardware that accesses data indicatingcurrent program states for a device or user, evaluates data receivedfrom the device or user or from other sources, and sets program statesfor the management plan of the device or user as a result of theevaluation. The program state setting module may perform theseoperations iteratively, e.g., periodically, for example every. Asanother example, the program state setting module may re-evaluate theprogram states for a device or user in response to receiving additionalinformation, such as sensor data, user input, data characterizing theoperation of a device, or other input. As discussed below, the programstate setting module may set the program state for one or more of theprograms based on the program states of one or more other programs, sothat the program states are interdependent.

The server system 110 can include an instruction generation module thatgenerates customized instructions to alter the operation of the manageddevices according to the updated program states for each device. Theinstruction generation module can include one or more software modulesthat access device-specific or user-specific program states for adevice, determine appropriate rules from the programs that correspond tothe states, apply the determined rules (e.g., evaluate a subset of therules for a program to determine which rules have conditions andtriggers satisfied), and generate instructions for the device to carryout system actions indicated by the rules. The rules that are applicableto an active segment of an active program can be evaluated using thecontext information historical information and other data. Theinstructions generated may cause any of a variety of changes in theoperation of a device, including, but not limited to, initiating andreporting a measurement with a sensor of the device or a connecteddevice, opening or closing an application, increasing or decreasing afrequency of sensor measurements made or outputs made by a device,switching an operating mode (e.g., entering or exiting a low-powermode), altering the appearance of a user interface, presenting certaininformation or media by the device (e.g., audibly, visually, orhaptically), altering the set of interactive controls made presented onthe device, altering threshold or triggers that the device applies,initiating a communication session between the device and anotherdevice.

The instruction generation module can operate using any of a variety oftechniques. For example, the instruction generation module may determineappropriate program levels, identify content (e.g., media, templates,sets of potential instructions, etc.) corresponding to those levels, andthen generate customized instructions as a selection from the identifiedcontent. The instruction generation module may filter or narrow thecontent for specific program levels based on the prior history for adevice or user (e.g., based on effectiveness of prior instructions,based on current and prior context information for the device or user,to avoid repeating instructions given recently, and so on.) As anotherexample, the instruction generation module may access a machine learningmodel for generating instructions, provide an indication of the programstates for a device or user to the machine learning model, and receiveoutput indicating types of content or instructions likely applicablebased on the program states. The instruction generation module may usethe output of the machine learning model to retrieve or generate dataelements that are sent in the customized instructions. As anotherexample, the instruction generation module may access one or moretemplates for instructions and populate the templates with elementsselected based on the device-specific or user-specific program statesfor a device and based on the current and historical information about adevice or its user.

The server system 110 can include a transmission module to sendcustomized instructions to devices. For example, the transmission modulemay include a network interface controller, such as a LAN adapter, WANadapter, network interface card, or other electronic circuitry(potentially with associated firmware and/or software) configured tocommunicate using a physical layer and data link layer standard such asEthernet or Wi-Fi. As customized instructions are determined for thevarious devices, the server system 110 sends instructions causing thedevices to vary their operation and outputs using the transmissionmodule, over a network, such as the Internet, a local area network, awide area network, etc.

FIG. 2 shows an example management system 200. The management system 200is an example of a system implemented as computer programs on one ormore computers in one or more locations (e.g., one or more serversystems) in which the systems, components, and techniques describedbelow are implemented.

The management system 200 selects operating plans, or a progression ofprograms, to be applied in order to manage devices within complex system100 determined based on training/feedback data 230. That is, themanagement system 200 receives training/feedback data 230, such ascurrent program states 214 of the devices of complex system 100, sensedconditions 216 of the devices of complex system 100, and/or outcome data212 of the devices of complex system 100. In response to thetraining/feedback data 230, the management system 200 generates a globaloperating plan output using the global operating plan neural network210. In some examples, each observation includes raw sensor datacaptured by one or more sensors, such as visual data, inertialmeasurement unit (IMU) readings, and so on. The training/feedback data230 can be specific to a particular device 120 a, 120 b, and 120 c ofthe complex system 100. In some examples, the training/feedback data 230is applicable across the complex system 100 generally, allowing themachine learning model(s) of the system 200 to discover and re-createresponse patterns that transition from one desired set of programs andlevels to another more desirable set of programs and levels.

The global operating plan neural network 210 is a neural network that isconfigured to receive an observation in the form of input data, such astraining/feedback data 230 and to process the observation to generate aglobal operating plan output in accordance with current values of theparameters of the global operating plan neural network 210. The globaloperating plan output defines a probability distribution over a set ofpossible actions to be performed in order to manage devices within thecomplex system 100, e.g., device 120 a, 120 b, and 120 c. For example,the global operating plan output may include a mean action vector andcovariances of the entries of the mean action vector. In this example,the global operating plan output includes a mean action vector thatincludes a respective entry, i.e., a respective mean spindle speed, foreach device tool and covariances of the entries of the mean actionvector.

To effectively tailor respective operating plans for particular devicesin response to training/feedback data 230, the management system 200trains the global operating plan neural network 110 to determine trainedvalues of the parameters of the global operating plan neural network110. The global operating plan neural network generates an operatingplan 220 a, 220 b, and 220 c for each of the respective devices 120 a,120 b, and 120 c. Each of the operating plans includes a progression ofprogram state settings for the particular device.

Generally, the management system 200 performs multiple iterations of atwo-step training approach to train the global operating plan neuralnetwork 210.

Instead of directly learning the parameters of the global operating planneural network 210, in the first step of the training approach themanagement system 200 uses a progressive, or trajectory-centric,algorithm to learn simple operating plans for a particular device (e.g.,device 102 a, 102 b, or 102 c) of the complex system 100. The globaloperating plan neural network 110 is trained based on feedback 230 fromthe operating plans 220 a, 220 b, and 220 c. Feedback 230 includes datasuch as outcome data 212, current program states 214, sensed conditions216, and historical data 218.

Outcome data 212 includes data indicating the outcomes of theapplication of the operating plans to respective devices. For example,whether a particular operating plan effected an improvement can berecorded as outcome data 212. Outcome data 212 can be generated orcollected by the global operating plan neural network 210. In someexample, outcome data 212 can be received from the operating plans 220or the devices 120.

Current program states 214 include data that indicates the status ofeach program as applied to each of the respective devices at the timefeedback 230 is collected. Current program states 214 can also includefeature vectors that represent a respective device. For example, afeature vector for a particular device can indicate that it is 8 yearsold, and has poor battery life. Sensed conditions 216 include datacollected by, for example, sensors (e.g., temperature sensors, digitalscales, optical sensors, cameras, inductive sensors, motion sensors,microphones, ultrasonic sensors, etc.). Sensed conditions 216 caninclude external conditions, such as the temperature, an air qualityindex, a speed, a direction, etc. or internal condition of a device.Current program states 214 and sensed conditions 216 can be receivedfrom the operating plans 220 or the devices 120.

Historical data 218 includes historical values for various feedback,including historical values for outcome data 212, current program states214, and sensed conditions 216. Historical data 218 can be collected andstored in a storage medium by the global operating plan neural network210. In some examples, historical data 218 can be stored in a storagemedium by the operating plans 220 or devices 120.

In particular, for each one of these devices 120 a, 120 b, and 120 c,the management system 200 generates an operating plan by selectingprograms to be applied in order to manage the devices 120. The featurevector includes the various programs and levels for which In particular,for each device, the management system 200 determines the state-to-statetransition, or operating plan, using the global operating plan neuralnetwork 210 and in accordance with the current operationalcharacteristics of each device. The global operating plan neural network210 receives, for example, a feature vector representing a particulardevice. The system can generate an operating plan in accordance withcurrent values of the parameters of the global operating plan neuralnetwork 210. That is, the system can receive a sequence of observations,or input data (e.g., training/feedback data 230) and, in response toeach observation in the sequence, process the observation using theglobal operating plan neural network to generate a global operating planoutput for the observation in accordance with current values of theparameters and then sample an action from the distribution defined bythe global operating plan output.

The management system 200 then optimizes a respective operating plan 220a, 220 b, or 220 c for each device 120 a, 120 b, or 120 c. That is, foreach device, the management system 200 optimizes an operating plan thatis specific to the device on the progression of programs and programlevels for the device.

In the second step of the training approach, after optimizing theoperating plans 220 a, 220 b, and 220 c, the management system 200 usesthe optimized operating plans to create a training set for learning acomplex high-dimensional global plan in a supervised manner. That is,the management system 200 generates training data for the globaloperating plan neural network 210 using the optimized operating plans220 and trains the global operating plan neural network 210 on thetraining data to adjust the current values of the parameters of theglobal operating plan neural network 210, e.g., using supervisedlearning. The management system 200 can train the global operating planneural network 120 using only received observations (e.g.,training/feedback data 230), and, thus, the global operating plan neuralnetwork 210 is able to predict actions from the training/feedback data230.

The management system 200 can effectively use the global operating planneural network 210 to select programs to be applied in order to managethe devices 120 a, 120 b, and 120 c once the global operating planneural network 210 has been trained. In particular, when an observationis received, the management system 200 can process the observation usingthe global operating plan neural network 210 to generate a globaloperating plan output in accordance with the trained values of theparameters of the global operating plan neural network 210. Themanagement system 200 can then select a new program or program level toapply to manage the devices 120 a, 120 b, and 120 c in response to newtraining/feedback data 230. For example, if training/feedback data 230,specific to device 120 c, is received, the global operating plan neuralnetwork 210 can process the training/feedback data 230 and learnglobally across the complex system 100, as well as apply an actiontrajectory specific to device 120 c.

In some implementations, the global operating plan neural network 210includes a convolutional sub-network (e.g., as an initial processingportion of the network) and a fully-connected sub-network.

The management system 200 can train the global operating plan neuralnetwork 210 on the training data to adjust the current values of theparameters of the global operating plan neural network 210 by, in someexamples, adjusting current values of the fully-connected parameterswhile holding current values of the convolutional parameters fixed. Insome other examples, the system 200 can train the global operating planneural network 210 on the training data to adjust the current values ofthe parameters of the global operating plan neural network 210 byadjusting current values of the fully-connected parameters and currentvalues of the convolutional parameters.

In some implementations, the system 200 pre-trains the convolutionalsub-neural network with a proxy pose detection objective to determinepre-trained values of the convolutional parameters, i.e., prior totraining the neural network 210 using the two-step approach.

Additionally, in some implementations, in addition to or instead ofpre-training the convolutional sub-neural network, the system 200pre-trains the neural network 210 on training data generated as a resultof conventional global policy search with operating plan optimizationperformed for each device 120 a, 120 b, and 120 c.

The system can repeatedly perform the training process until terminationcriteria for the training of the global operating plan neural networkare satisfied to determine trained values of the parameters of theglobal operating plan neural network. For example, the system canperform the training process for a particular amount of time, until acertain number of iterations of the training process have beenperformed, or until the global operating plan neural network 210achieves a threshold level of performance across the operating plans220.

FIG. 3A is a diagram that illustrates an example of a generalized system300 of network connected devices for generating dynamic, customizedmanagement plans for each of the devices. The system 300 includes aserver system 310, client devices 320 a, 320 b, and 320 c, anadministrator system 330, and additional systems 440 a, 440 b, and 440c. The server system 310 communicates with each of the other componentsof the system over a network 316. The server system 310 also accesses anumber of data collections, illustrated as databases 311, program rules312, and program content 313.

The example in FIG. 3A shows various interactions that can be used tocreate, adjust, and carry out management plans. As described above, thesystem 300 can generate management plans for a variety of complexsystems. The situation in which a user is the complex system for whomthe management plan is generated is described solely for ease ofillustration. As an example, interactions shown in stages (A) to (E)follow a process of creating a management plan, monitoring the device'sstatus and needs, providing instructions of commands, and adjusting themanagement plan for each device. The management plan maintained andadministered by the system is not merely a statement of desired deviceperformance or operating plans. Rather, the management plan provided bythe system 300 is a real-time, interactive customized service thatinitiates communication and interaction with the managed device and/orthe device's user to adjust operation of the device. Among othereffects, the management plan can generate instructions, initiated ortriggered by the system 310 in response to sensor data or other detectedevents, that change the device's performance or mode of operation,including various outputs of the device.

During stage (A), the server system 310 generates and provides anadministrator portal 332, which may be provided as a web page, webapplication, data for a locally executable application of theadministrator system 330, or in another form. This interactive portal332 enables the administrator to view management plan templates 334 andselect a template to use to enroll a new device or user. The portal 332also provides controls for the administrator to modify the templates 334for individual users, as well as view and alter management plans thatare currently active for any of the users assisted by the administrator.Examples of other information that can be provided include indicationsof performance targets (e.g., goals) and progress over time as shown inFIG. 3E.

The administrator portal 332 provides a number of management plantemplates 334 that the administrator can select when enrolling a newdevice or user. In some instances, the templates correspond to differentareas or fields that a user wishes to manage. The management plans canbe generated as combinations of different digital operating programs. Atemplate 334 can specify a set of these programs that are made inactivefor a given device or user. The programs each have a number ofcomponents and states that very how they interact with the device oruser. For example, each program may include multiple segments that areapplicable over different time periods. The program can define asequence of these segments, or multiple different sequences. Thesegments may correspond to fixed length periods, for example, a week, ormonth, or another fixed or standardized time period. Alternatively, oneor more of the periods may have a variable duration.

Each program has various levels or intensities with which the programapplies. The level of the program may affect, for example, howfrequently the program communicates with the device or instructs thedevice to present output to a user, how ambitious or difficult the goalsand activities of the program are, how closely behavior is monitored,and so on.

When a device or user is first enrolled, the management plan template334 selected by the administrator specify a default set of programs andcorresponding levels for the user. The administrator may manually adjustthe selections, for example, by adding or removing programs, andincreasing or decreasing the levels of programs. In this matter, theadministrator can specify an initial management plan that meets theneeds identified by the device, user, or other users, such as the user'ssupport system.

With the interfaces and tools provided by the server system 310 throughthe administrator portal 332, an administrator can efficiently managemanagement plans for a large number of user. In many instances, afterthe management plan is initiated, the administrator does not need tofurther modify or adjust the management plan. The server system 310automatically adjusts each individual management plan according to theunique set of user inputs, sensor data, and behavior detected for eachdevice or user. Nevertheless, an administrator can view an individualdevice or user's progress and, if appropriate, adjust a device or user'smanagement plan manually through the administrator interface 332. As theserver system 310 adjusts the management plans based on each device oruser's individual circumstances, different management plans will havedifferent segments of programs, different levels of the programs, anddifferent combinations of the programs active. These differences as wellas the application of individual program rules to unique device or userdata sets, provide a customized progression of operating states to eachdevice and a unique experience to each user.

During stage (A), the server system 310 stores information about eachdevice or user enrolled in the system. As discussed further below, theserver system 310 stores data indicating the current state of a deviceor user's management plan, such as the programs that are active, thelevels of each program, which segment the user is currently in for eachprogram, and so on. In addition, the server system 310 stores historicalinformation about the device or user and its/his interactions with theapplication and progress while using the management plan. The serversystem 310 can acquire data from many different sources includingsurveys, sensor data, and other data discussed further below. Thisinformation can be stored in various databases 311, which are shown asone example of a data storage subsystem.

The server system 310 can store the data using a data storage subsystemwhich may include, for example, direct attached storage (e.g., hard diskdrives, solid state drives, etc.), locally attached storage, networkattached storage (NAS), one or more databases, a storage area network(SAN), a database management system (DBMS) and/or one or more databaseservers, or other elements. For example, the data storage subsystem mayinclude one or more relational databases (e.g., an SQL database), one ormore object-oriented databases, one or more noSQL databases (e.g., akey-value database, a wide column store database, etc.), a distributeddata store, one or more file systems, or other data storage system. Thisdata storage subsystem can store data gathered from devices beingmanaged (e.g., state information, sensor data, operation history, etc.)as well as information that the server system 110 generates (e.g.,current and prior levels for different programs, instructions todevices, source content and generated content to be transmitted todevices, etc.).

During stage (B), the server system 310 also stores information thatdefines each of the programs. Each program can include a correspondingset of program rules 312. A repository of these rules 312 is maintainedand accessed by the server system 310, and the rules are evaluated on anongoing basis to determine when to communicate with each device or userand in what manner. If a plan has multiple programs active, rules foreach program separately can specify different communications orinterventions to be provided to the user. Some examples include alteringoperation of a device, initiating a scan of an environment nearby,acquiring data indicating a state of a device, discovering and issuingcommands to nearby devices, providing media to a user (e.g., text,audio, video, etc.) using the output devices of a device (e.g., displayscreen, speaker, vibrator, etc.), generating an interactive form such asa survey for the user, sending a notification email or other message,challenging, reminding, or informing the user about a goal, providingrecommendations, providing content from a social media platform,providing instructional activities or games, and so on. These and otherinteractions can also be provided to friends, family members, medicalservice providers, and others to support the user. For example, surveysor recommendations can be generated and sent to friends and family of auser, to corroborate conditions indicated by the detected behavior orinteractions of the user. In this manner, the various programs cansupport a user in some way, through interactions with one or more otherassociated users of the system. In some instances, these types ofinteractions may provide better results than changing the operation ofthe user's device.

The rules for the programs have conditions and triggers associated withthem. Each rule may have a condition that must be satisfied for acorresponding system action of the rule to be performed. In addition, arule can have a trigger or triggering condition associated with it, sothat the action of the rule occurs when the trigger is satisfied as wellas one or more other associated conditions.

The actions of the server system 310 to generate device-specific oruser-specific interfaces and content can draw from a repository ofprogram content 313. This repository 313 can include media, messages,and other content that is selectively provided according to the programrules in the repository 312, which in turn are selectively usedaccording to which programs are active, which levels of the programs areactive, and which segments of the programs are active.

During stage (C), the server system 310 obtains data for each of thevarious users enrolled with the management plan. The server system 310communicates with devices 320 a, 320 b, and 320 c to acquire data foreach of the various users in rolled with the management plan. The serversystem 310 communicates with devices 320 a, 320 b, and 320 c to acquirecontext information, such as the location of the device, a currentactivity of the device, sensor data, and other data. Each of the devices320 a, 320 b, and 320 c can have a respective user for whom themanagement plans apply. The server system 310 also communicates withthird-party systems to access data such as user activity andinteractions, and even general information such as current news,weather, and other data.

In stage (D), the server system 310 applies the individual managementplan for each device or user to the data collected for the device oruser. For example, the server system can include an instructiongeneration module that generates customized instructions to alter theoperation of the managed devices according to the updated program statesfor each device. The instruction generation module can include one ormore software modules that access device-specific or user-specificprogram states for a device, determine appropriate rules from theprograms that correspond to the states, apply the determined rules(e.g., evaluate a subset of the rules for a program to determine whichrules have conditions and triggers satisfied), and generate instructionsfor the device to carry out system actions indicated by the rules. Therules that are applicable to an active segment of an active program areevaluated using the context information historical information and otherdata. The instructions generated may cause a variety of changes in theoperation of a device, including, but not limited to, initiating andreporting a measurement with a sensor of the device or a connecteddevice, opening or closing an application, increasing or decreasing afrequency of sensor measurements made or outputs made by a device,altering an operating mode (e.g., entering or exiting a low-power mode),altering the appearance of a user interface, presenting certaininformation or media by the device (e.g., audibly, visually, orhaptically), altering the set of interactive controls made presented onthe device, altering threshold or triggers that the device applies,initiating a communication session between the device and anotherdevice.

The instruction generation module can operate using any of a variety oftechniques. For example, the instruction generation module may determineappropriate program levels, identify content (e.g., media, templates,sets of potential instructions, etc.) corresponding to those levels, andthen generate customized instructions as a selection from the identifiedcontent. The instruction generation module may filter or narrow thecontent for specific program levels based on the prior history for adevice or user (e.g., based on effectiveness of prior instructions,based on current and prior context information for the device or user,to avoid repeating instructions given recently, and so on.) As anotherexample, the instruction generation module may access a machine learningmodel for generating instructions, provide an indication of the programstates for a device or user to the machine learning model, and receiveoutput indicating types of content or instructions likely applicablebased on the program states. The instruction generation module may usethe output of the machine learning model to retrieve or generate dataelements that are sent in the customized instructions. As anotherexample, the instruction generation module may access one or moretemplates for instructions and populate the templates with elementsselected based on the device-specific or user-specific program statesfor a device and based on the current and historical information about adevice or its user.

Among other features discussed above, the instruction generation moduleallows the system 310 to determine, for each particular program and eachparticular device or user at the current time, what information shouldbe requested through a survey perhaps or what information should beprovided to the device or user. The instruction generation can be donefor each of the programs that are active in the device or user'smanagement plan, or holistically for the combination of programs as awhole. For example, the server system 310 can take all of the results ofthe different programs (e.g., changes in operation of the deviceindicated by those programs) in the management plan and generatecustomized user interface data to be sent to the appropriate device ordevice of the user. Although different programs may indicate differentmessages to be provided or different survey questions to present to auser, the server system 310 may integrate the content across the variousprograms.

As the user interface data is provided to the devices 320 a, 320 b, and320 c, users interact with the digital management plan application ontheir devices 320 a, 320 b, and 320 c. Data indicating inputs to theapplication and the manner in which the user acts in response to thecommunication are determined and provided to the server system 310,which logs this information. For example, if a user is shown a messagerecommending they go outside and take advantage of the good weather, andsubsequently does go outside as detected by motion sensors and locationsensors of the device, the server can store this information and use itto determine future content for the user. Examples of user interfacesshowing user interfaces that may be provided on the user devices 320 a,320 b, and 320 c are shown in elements 350 a, 350 b, and 350 c, whichare shown in an expanded view in FIGS. 3B, 3C, and 3D.

The server system 310 can include a transmission module to sendcustomized instructions to devices. For example, the transmission modulemay include a network interface controller, such as a LAN adapter, WANadapter, network interface card, or other electronic circuitry(potentially with associated firmware and/or software) configured tocommunicate using a physical layer and data link layer standard such asEthernet or Wi-Fi. As customized instructions are determined for thevarious devices, the server system 310 sends the instructions to causethe devices to vary their operation, e.g., to change their outputsand/or internal functioning, using the transmission module, over anetwork, such as the Internet, a local area network, a wide areanetwork, etc.

FIGS. 3B, 3C, and 3D illustrate that the user interface providesdifferent user-selectable controls, such as buttons, links etc., anddifferent content in the user interface of a device. For example, inFIG. 3B, when contact information, such as a telephone number, is shown,an action button to place a call is available. In FIG. 3C, the userinterface reflect that the device was instructed to acquire data fromsensors of the device, such as an accelerometer. In FIG. 3D, the devicewas instructed to acquire information from another device, in this casea pulse oximeter and an EKG sensor, and the device performs thoseactions and provides the results to the server system 310.

During stage (E), the server system 310 adjusts the states of theprograms if appropriate. The server system 310 can include a programstate setting module to evaluate and change the states of programs ineach management plan. Using the program state setting module, the serversystem 310 can assess various factors to determine whether the set ofprograms that is active should change, and whether the levels and activesegments of each program should change. In some instances, a device oruser progresses through segments of a program based on the passage oftime. The program state setting module of the server system 310 candetect when an appropriate transition should occur from one time periodor one section of a program to another. In general, the program statesetting module can include software that accesses data indicatingcurrent program states for a device or user, evaluates data receivedfrom the device or user, and then sets program states for the device oruser to use going forward. The program state setting module may performthese operations iteratively, e.g., periodically, for example every. Asanother example, the program state setting module may re-evaluate theprogram states for a device or user in response to receiving additionalinformation, such as sensor data, user input, data characterizing theoperation of a device, or other input. As discussed below, the programstate setting module may set the program state for one or more of theprograms based on the program states of one or more other programs, sothat the program states are interdependent.

Transitions between states or levels of programs can be based onmeasures of performance determined by or entered to a device, which canbe assessed with respect to certain targets for goals, which may bestandardized or maybe device or user specific. For example, a programfor physical activity may set a goal of a number of steps per day forthe user to walk. When a user successfully completes ago the system maydetermine that a level corresponding to a higher goal is appropriate. Insome instances, a higher level represents a need for greaterintervention or monitoring. As a result, or user who successfullycompletes goals for physical activity may be transitioned automaticallyto a lower level of the physical activity program in which fewerreminders or interactions are provided, since the user has shown theability and willingness to exercise. On the other hand, are user who isnot exercising to the extent needed, or who is not responding to theinterventions of the program may be transitioned to a higher levelindicating more intense or different types of interaction in an attemptto improve the users physical activity.

Thus, by varying the level of each program as management proceedsthrough different program segments, the server system 310 can tailor thedegree of interaction and intensity of support across various aspects ofthe user's wellbeing and lifestyle.

When adjusting the programs in a management plan, the levels or statesof the programs can be interdependent. Changes in the level of oneprogram may result in changes to other programs. These interdependentlevel transitions may depend on more than simply the levels of otherprograms, and maybe based on any of the data collected from or about auser as well, as well as any of the outputs of the different programs.

One of the ways that the server system 310 interacts with users is toprovide interactive forms and surveys. Each form can be customized ordynamically created for a user's current status and the current state ofhis management plan. These forms can request information that isindicative of the user's progress with respect to one or more activeprograms in the management plan. Similarly, the surveys may includequestions or other interactive activities that would enable the serversystem 310 to evaluate whether additional programs should be activated.The server system 310 may use a variety of computer adaptive testingtechniques to generate forms that acquire sufficient information neededto determine her use of progress for various programs, withoutoverburdening the user.

As an example, each of the programs that are active in a management planmay indicate, periodically or intermittently, types of data needed froma device or user. Even for a single program, the type of data needed mayvary over time. For example, different segments of a program maycorrespond to different combinations of topics. Different levels mayrequire different frequencies of input or different precision of input.Thus, each program may provide a list of data items to obtain from auser. The server system 310 and may assess these lists to generateappropriate interactions that obtain the appropriate information andbenefits the user.

The server system 310 may de-duplicate the set of data items so thatmultiple redundant questions are not asked. The server system 310 alsochecks recent answers and interactions of the device or user, as well asany other data sources (e.g., location data, sensor data, onlineactivity, communications of family and friends with the server system,etc.) to determine if the requested information is available from othersources already. For the remaining items that are not yet available, theserver system 310 may prioritize or schedule the data acquisition toreduce the overall burden of any one interaction and increase thelikelihood that the user will complete the form successfully.

The server system 310 may group the remaining data items into relatedgroups. For example, the data needed by multiple programs may beacquired through a single form, and in some instances multiple items ofdata may be able to be obtained through a single question.

The server system 310 can access a database of questions and interactiveactivities, with each item having metadata specifying the types of dataobtained through the question or activities. Thus, by mapping neededdata types to the various types of interactions in the database, theserver system 310 selects a few questions or interactions that canacquire the needed input.

The server system 310 may implement the survey or form generationprocess to allow significant flexibility. Individual programs mayprovide fully formed questions or interactive elements to beincorporated into a user interface. In addition, or as an alternative,programs may simply provide an indication of data types to be acquired,and the system 310 can formulate appropriate questions and userinterface elements for obtaining the data. As noted above, the serversystem also may combine and integrate the requests needed by variousprograms.

In many cases, the questions and interactions in a survey are adapted tonormal daily life of the user. For example, valuable information can beobtained through journal entries a user enters in association with theapplication. The application can ask about the user's plans, recommend afriend to reach out to, assist with goals and tracking, and generallyprovide daily assistance. Although these interactions may not relatespecifically to the user's goals, the content provided to a user anddata obtained from the user in this manner can be used to carry out andupdate the management plan.

The data the server system 310 obtains can directly affect whichinterventions or interactions are provided, which programs are activeand to what extent (e.g., what levels are set for the programs), theassessment of various risk levels, and so on.

FIG. 4 shows an example of a system 400 for providing a digital actionmanagement plan. As discussed further below, the server system 310 isable to adjust and adapt the management plan for a device or thedevice's user, and also provide direct interventions and supports toencourage achievement of the goals of the management plan.

In the example of FIG. 4, a user has a mobile device 420. The device 420or its user has previously been enrolled to a management plan by anadministrator. Mobile device 420 has an application installed that helpscarry out the interactions and interventions of the management plan. Forexample, the application 421 may collect sensor data and user input fromthe device 420 and other devices nearby on an ongoing basis. Theapplication 421 provides this data to the server system 310 for storageand analysis. As the server system 310 applies the appropriate rules andprograms of the user's management plan, the server system 310periodically sends customized content and interface data to the mobiledevice 420 to be displayed through the application 421 or interacts withthe mobile device 420 in another manner.

In general, the management plan can push content to the mobile device420 for display as triggered by the rules of the various programs activein the management plan. Rather than the user manually requestingcontent, e.g., by opening the app to navigate to a web page, the serversystem 310 and/or the app 421 can initiate intermittent and unexpectedinteractions throughout the course of a day, week, etc. In a sense, theinteractions of the management plan can be generated through a kind ofalways-on or continuous analysis by the server system 310. The programsof a management plan can provide contextually relevant interactions atany time, as determined appropriate by the server system 310 from thevarious data sources available to the server system 310.

The server system 310 communicates with the mobile device 420 over thecommunications network 316. To carry out the management plan, the serversystem 310 may also communicate with various other devices andthird-party systems.

The server system 310 also interacts with third-party systems,illustrated as computing systems 426 and 428. These systems 426 and 428represent any of a variety of public and private data sources.Additionally, the systems can represent sources for other data, such asweather data, environmental data such as pollution levels, upcomingevents data, map data, and so on.

As a result, the server system 310 may interact with a first device,e.g., device 420 having a user. The system 310 can also interact with asecond device associated with a service provider. The system can alsointeract with a third device, e.g., device 422 or device 424, associatedwith another user, such as a co-worker, friend, family member, etc.

With input collected on an ongoing basis from these data sources andothers, the server system 310 applies the rules of each active programin the user's management plan. These rules can specify various differentsystem actions to be performed. The server system 310 can administerprograms and evaluate rules using any of the techniques discussed inU.S. Pat. No. 9,848,061, filed Dec. 19, 2017, and/or U.S. Pat. No.9,753,618, issued on Sep. 5, 2017, both of which are incorporated hereinby reference.

The incoming data stream regarding a device user, compiled from allavailable data sources, may be used to form a current data set orprofile for the device or user. The server system 310 continuouslyupdate the profile representing the current status of the device oruser, using all of the various input data sources.

Among the various actions that the server system 310 can perform areactions that alter or adjust the combination of programs in a managementplan and the program state or level of individual programs. For example,the server system 310 may evaluate the incoming stream of data about adevice or user, and extract specific data types and measurementsrelevant to each program. The rules that are applicable at a given timecan specify the types of data that are needed for a program. Forexample, of the rules of a program, only a proper subset of the rulesmay be applicable to the current level and segment of the program. Theserver system 310 can identify the set of data needed to determinewhether the conditions and triggers of the rules in the subset aresatisfied.

The different program states or levels of a program can representdifferent portions, configurations, or operating characteristics of aprogram. Accordingly, changing the program state for a program can causethe system to use a different portion, configuration, or operatingcharacteristic. In general, different program states may correspond todifferent levels of magnitude or frequency for changing operation of adevice. Different program states may also cause the program to usedifferent sets of content to provide to a device, or may use ordifferent sets of rules to determine when and how to adjust operation ofa device. Similarly, different program states may alter applicationbehavior in various ways, such as by altering the level (e.g., type,magnitude, or frequency) of interaction with a user, or altering thetypes of options that are presented to a user on an interface of adevice. In some implementations, different program states may correspondto different levels of targets or goals for the performance of a deviceor user. For example, at a first level or program state, a modest targetis set, with more aggressive goals being set for other program statesrepresenting a need for a greater magnitude of change, or a greaterurgency or frequency of making changes. In a similar manner, differentprogram states may set different thresholds or triggers for deviceoperation, and/or may cause different code of the programs to be used orcause different instructions to device to be generated.

Each program may have, in its corresponding set of rules, one or morerules setting conditions and triggers for altering a program state.These factors may be unique to each program, and to each leveltransition. That is, a first program may have different sets of rulesthat govern whether to transition from level 1 up to level 2, from level2 up to level 3, from level 3 down to level 2, and so on. In someinstances, changes in program state may occur at any time thatappropriate conditions and triggers are met. In other instances, changesin program state may occur at transitions between segments or at otherspecified intervals.

In the example of FIG. 4, charts showing program states over time areshown for three different programs 440 a, 440 b, and 440 c in themanagement plan for a device or user. The program states vary over timedue to the server system 310 determining that more or less interventionis needed for corresponding topics at different times. The transitiondecisions are made based on rules of the programs applied to the currentstatus profile for the device or user at the specific point in time whenthose transitions were made. As illustrated, the state of the firstprogram 440 a varies significantly over time. This may represent forexample, variations for a power management program responding to highlyvariable loads experienced by a device. The second program 440 b shows agradual decrease from level two, to level one, and then disabling theprogram. This may represent, for example, steady progress of the devicein managing a resource, such as communication bandwidth, to the extentthat the rules of the program specify that no further monitoring isneeded. The third program 440 c represents a constant level over thetime period. This may indicate, for example, that a device or user ismaintaining acceptable performance, but may not be meeting targetperformance levels sufficiently to reduce the level.

The programs can be interdependent. The state of any given program maydepend on the state of one or more other programs, or even all otherprograms in the management plan. Various techniques may be used toprovide this interdependence. For example, program rules that specifysystem actions to be performed under certain conditions may specify thatcertain states of certain programs are a condition or trigger for alevel transition. Similarly, the act of making a program active in amanagement plan or transitioning to a different level of a program maybe a condition or trigger.

As another example, the output of a program may be used to setlevels/program states of other programs. For example, the server system310 can assess the types of survey questions or data types that programsuse. Each program may include a specification of data types that areassessed using the program, or the specification can be inferred fromthe rules of the program. If a program is disabled but has aspecification indicating data types that match those being requested byother programs, the disabled program may be activated. In someinstances, analysis of output by a device or the results of some of thevarious programs, and data input requests of other programs, can be usedto determine a relevance score between current a current state of adevice or user and the other programs. When the relevance scoresatisfies a threshold, this may trigger the activation of a program oran adjustment to the level of a program.

In some implementations, relationships between programs may bepredetermined, e.g., manually defined or explicitly set through rules.This is not required, however. Indeed, the analysis of the combinedprogram states in a management plan and/or the outputs of the programsto a user can be effective to adjust program states even with nospecific knowledge of interconnections between the subject matter ofdifferent applications/programs. The server system 310 can inferrelationships through commonalities in program specifications andoutputs of the programs. That is, the patterns of outputs of a programand requests for inputs by the program can show over time in the mannerin which the program relates to other programs. The server system 310may analyze these relationships, for example, comparing which segmentsof programs and which levels of programs relate to specific levels andsegment of other programs. The system 310 can also apply machinelearning techniques, such as reinforcement learning, to patterns ofmanagement plan creation, level transitions over time, and otherfactors. With a sufficient data set generated from the progress of manyusers, the server system 310 can identify typical patterns in userstatus profiles, with the corresponding combinations of programs andlevels. As the system 310 assesses program state transitions over time,the server system 310 is able to identify combinations of programs andlevels that produce the best results for users with specificbackgrounds. For example, the server system 310 may identify, for eachof multiple different cluster of negative outcomes or undesired states,which combinations of programs, and at what levels, are most effectiveat reducing the negative outcomes or undesired states over a timeperiod, e.g., a week, a month, a year, etc.

For example, the server system 310 may evaluate user status profiles andcorresponding program states over a period of time, and determine thatusers that have three particular programs active simultaneously have thehighest likelihood of achieving a decrease in the level of a fourthprogram. From this determination, the server system 310 may generaterules that activate the three programs together if the fourth program isactive. More complex conditions may be set if desired. For example,rather than activate all three programs, the change may be limited toactivating only one of the programs if two of the three are alreadyactive and the fourth is also active. As another example, all threeprograms in the group may be activated if the level of the fourthprogram is above a threshold, but not if the level of the fourth programis below the threshold. Of course, the behavior learned of the serversystem 310 does not need to be expressed in explicit rules and can, invarious implementations, be incrementally learned as part of a trainingstate of a classifier, neural network, or other machine learning model.

In general, the server system 310 can use a variety of interactions,measurements, testing, feature vectors and stated or observed behaviorsof a user to set program states. The server system 310 can use thesesame factors to select which digital management plan to use for adevice, and when and how to provide instructions that alter theoperation of the device. In addition, the server system 310 can identifydigital state markers and use them as indicators for selecting certainmanagement plans.

The server system 310 may evaluate various data sets to determine whichcombinations of data about a device or user constitute a digital statemarker that is predictive of certain outcomes, whether desirable orundesirable. These digital state markers may represent markers forfuture performance of a device or user. As another example, digitalstate markers can be predictive of future outcomes for a device or user.For example, the server system 310 may access data sets for longitudinalstudies that show data collections describing many devices, theircharacteristics, and outcomes over time. From analysis of this data, theserver system 310 can select combinations of characteristics and actionsthat satisfy a minimum threshold of relevance to different outcomes, anddefine the identified combinations of data factors as digital statemarkers. As a result, the system may detect combinations of programstates that serve as a digital state marker for positive outcomes (suchas a device's efficiency, output quality, long uptime, highresponsiveness, maintained security, etc.) or for negative outcomes(e.g., virus infection, insecurity, bandwidth bottlenecks, overheating,etc.).

In some implementations, digital state markers can be defined in termsof digital operating programs and their states. As an example, thedigital state markers can be combinations of program states that governoperation of a device, and represent at least a minimum likelihood of aparticular event occurring for the device within a certain time frame(e.g., device failure within one year with at least an 80% likelihood,or running out of battery power in the next 30 minutes). Accordingly,the server system 310 may define these combinations of programs andstates as feature vectors that may be used to alter the combination ofstates, for example, to add a particular additional program to reducethe risk of a negative outcome, or increase the likelihood of a positiveoutcome. The digital state markers may also optionally be defined interms of (i) combinations of program states concurrently active at atime plus (ii) one or more attributes or conditions of a device or user.For example, one combination of program states may not represent amarker for a certain outcome on its own, but that combination may beconsidered a digital state marker when combined with certain detectedproperties of a device (e.g., battery less than 25% and CPU usageconsistently above 50%, or a certain device model and device age).

The server system 310 performs a variety of functions in creation andadministration of the dynamic, customized management plans. A few ofthese functions are represented in FIG. 4 as various modules of theserver system 310. While the various functions are illustratedseparately for clarity in illustration, various implementations maycombine these functions into a single module for a different grouping ofmodules.

A management plan update module 450 can modify the management plan foreach individual user, for example, based on changes in status of thedevice or user, or manual entry from an administrator through the portaldiscussed for FIG. 5.

A rule evaluation module 451 can select, from each active program of amanagement plan, the appropriate rules corresponding to the currentsegment and level of the program. Then, the rule evaluation module 451compares aspects of the status profile of the device or user with thetriggers and conditions of the selected rules. The rule evaluationmodule 451 causes the system actions corresponding to rules withsatisfied triggers and conditions to be performed.

A risk assessment module 452 can determine the unique risk profile for adevice or its user given the current profile and historical data.Because the server system 310 collects and stores information fornumerous environmental and historical factors, the server system 310 cancalculate the customized risks based on the data set compiled for thedevice or user. To aid in generating these risk levels, the serversystem 310 may store and access data sets representing outcomes andstatistics representing many different groups of people. From clinicaldata and statistical analysis of the data sets, the server system 310can determine a baseline risk level as well as a large set of factorsthat increase and decrease risk. The server system 310 identifies whichof the many factors are applicable given a user's current profile andhistorical data and adjusts the baseline risk accordingly. In thismanner, risk levels can be generated for many different conditions.

The risk assessment module 452 can determine a device's or user's riskwith respect to a set of potential outcomes and update those as theperson's profile changes. Similarly, as the server system 310 receivesnew assessment data or as outcome patterns are observed for the deviceand users affected by management plans of the system, the risk levelsare also changed. The risk levels determined by the risk assessmentmodule 452 may be used in a number of ways. In some instances, a personmay be informed of their risk level and how it compares to other user.As another example, the risk levels may be provided as input to therules of various programs, provided as input to the rules of variousprograms, which may condition they performance of certain system actionson risk levels in certain ranges. For example, a risk level for reducedmobility may be calculated and provided as an input to a program forphysical exercise, which may increase or decrease its level based onthat risk level. Risk levels for a user may trigger the activation of aprogram or increase in the level of a program, e.g., if the user's riskof experiencing an associated condition is above a threshold. Similarly,risk levels can trigger the deactivation of a program or decrease in thelevel of a program, e.g., if the user's risk is below a threshold.

The program state transition module 453 may govern transitions inprogram state for a person's management plan. As discussed above,changes may be made due to the interaction and execution of rules fordifferent programs. In addition, or as an alternative, a module of thesystem 310 may be used to manage these changes. This module 453 mayinitiate changes in programs based on analysis of a user status profileand current and previous program states for the user. The program statetransition module 453 may limit or normalize changes indicated byprogram rules. For example, the program state transition module 453 mayenforce certain restrictions, such as limiting a frequency of levelchanges that are allowed over a period of time. An advantage of usingthe program state transition module 453 to initiate program statetransitions is the ability to apply analysis for the management plan asa whole, rather than just as the rules of a single program within amanagement plan. As a result, the program state transition module 453can enforce various aspects desired for management plans, whetherdetermined using manually set rules, administrator preferences,relationships learned through analysis of stored data, and so on.

The data acquisition module 454 coordinates the requests for informationfrom devices 420, 422, 424, 426, 428, and others. This data acquisitionmodule 454 may obtain a list of requested data types or questions fromthe various programs, assess them to condense into a final set of neededinformation, and schedule interactions to acquire the needed data.Similarly, the data acquisition module 454 may periodically requestsensor data, weather data, and any other applicable type of informationneeded from various systems to allow the rules of the programs to beevaluated.

An instruction generation module 455 produces data packages that aresent to the application 421 on the mobile device 420. The instructiongeneration module 455 specifies content to be provided on the userinterfaces of the application 421, which may include variouscombinations of media, text, data entry elements, and so on. When amodule specifies that certain content should be provided, the interfacegenerator 425 receives the indication, determines the content layoutformatting and so on that are needed. Through the interfaces that aregenerated, the system provides many different types of interactions withusers. For example, the programs and other elements of the system 310may cause a survey to be provided, provide media to the user, provideinstructional materials, provide a test, provide a game, or initiateother activity with the user. Further, the instruction generation module455 can specify interactions beyond interactions with a visible display.Interactions can be specified to involve audio output, voice input,haptic output, gesture input, and other input/output modalities.

FIG. 5 shows additional data that the server system 310 can use togenerate dynamic, customized management plans. The figure shows threeprograms from FIG. 4, with an indication of stored data anduser-specific parameters for each.

Each program includes a set of program rules 520 a, 520 b, and 520 c.The overall set of rules for a program are generally applicable to allusers of the program. However, as discussed above, different subsets areapplicable to different users at different times. Further, of theapplicable subset of rules, only certain rules will have theirconditions and triggers satisfied at any given time. Rules may be markedwith metadata or may include elements specifying which levels andsegments of a program each rule corresponds to. This can allow theserver system 310 to filter, for each user, the overall set of rules forthe program to a smaller subset to actually evaluate for the user. Thiscan greatly increase efficiency and reduce computation, especially whenthe server system 310 supports a very large number of devices or userssimultaneously.

Each program includes a set of program content 522 a, 522 b, and 522 c.This content represents source material from which customizedinteractions with a user are generated. For example, the content for aprogram may include questions, videos, audio segments, images,instructional materials, messages (e.g., indicating encouragement,reminders, etc.), games, and other content. When indicated by theprogram rules 520 a, 520 b, and 520 c, specific portions of the contentmay be accessed, combined into a data package for the user, and providedby the server system 310 to the mobile device 520 for presentation tothe user.

Each management plan includes device-specific or user-specificparameters 524 a, 524 b, and 524 c for each program. This does not meanthat the set of parameters is unique among all users and devices;rather, each user or device has a separate set of parameters (e.g.,program levels and segments) that are separately stored and adjusted,and so are specific to that device or user. The parameters can includethe current level of the program, current targets or goals for theprogram, historical information for the device or user, and anindication of the current period or segment that is active in theprogram. In general, segments can represent to different sequentialperiods of time, in which the content or interaction of a program with adevice may change. The device-specific or user-specific parameters 524a, 524 b, and 524 c specify how the programs 440 a, 440 b, and 440 cprocess the input data 510, and ultimately how the customizedinstructions and interactive user interfaces 530 are determined.

The level can represent a level of need or intensity of neededinteraction for the topics or categories of that program. Thus, a higherlevel may indicate more severe need for monitoring or interaction whilecarrying out the management plan. This is optional, however, and levelsmay have a different meaning. In some instances, a high level mayindicate greater progress, for example, goals in an advanced range,achievement of certain skills, or other positive attributes.

The server system 310 is also shown receiving a set of input data 510.This input data 510 can be used to determine the current status of adevice or user, or historical information about the device or user.Input data can be actively requested by the server system 310 from adevice, another server system, or another device than the one associatedwith the management plan. Input data can also be passively received,e.g., periodically sent or broadcasted by other devices. As discussedabove, the input to the server system 310 can include may differentsources including sensor data from a mobile device or other devices,assessment data (e.g., survey responses, test results, etc.), input fromadministrators, weather data, points of interest data, or environmentaldata, data from a wireless wearable device, to name a few. Other typesof data used by the server system 310 in carrying out the programs 440a, 440 b, and 440 c and adjusting the states of the programs 440 a, 440b, and 440 c is discussed with respect to FIG. 8.

FIGS. 6A and 6B are diagrams that illustrate examples of techniques foradjusting states of digital operating programs.

FIG. 6A shows an arrangement 600 of elements in which each of variousprograms 440 a, 440 b, and 440 c receive an input data set used toevaluate their corresponding rules. These data sets may be the same foreach program or different for different programs. In addition toreceiving the input data representing a current status and historicalinformation for a device or user, each program receives a signal fromthe program state setting module 453. The module 453 assesses input data510 which may include any or all of the information used by the programs440 a, 440 b, and 440 c. The module 453 issues program stateinstructions which can dynamically change the level or state of eachprogram as the module 453 determines appropriate.

As the programs provide output to the instruction generation module 455,the output of the programs is also provided to the program state settingmodule 453. The output can represent messages to be provided, questionsto be asked in a survey, interactive activities of the application,indications of media provided to the user, and so on. The module 453 mayanalyze these program outputs and use the analysis results to change thecurrent levels of the programs. The program state setting module 453 cananalyze the outputs of the various programs in a number of ways. Forexample, the module 453 may extract text from questions or messagesinitiated by the programs, and map keywords and phrases in the extractedtext to different topics. Other mappings may indicate relationshipsbetween topics and programs, allowing output of one programs to beassessed for relevance to other programs. Sentiment analysis can also beperformed on the outputs of the programs, allowing the module 453 todetermine, for example, whether conditions are improving or worsening.The module 453 may also assess the frequency of activity of eachprogram, for example, determining whether a program is initiatingcommunication with increasing or decreasing frequency, and take thisinto account in assessing which programs and levels are likely to beneeded.

The module 453 also receives data indicating the current level andsegment that is active for each program, and this data also is used bythe module 453 to update the levels of the programs. As shown in FIG.6A, the program state setting module 453 may use information from any orall of the programs to set the state of any or all of the otherprograms. From data indicating the progress of various users over thecourse of using a management plan, the program state setting module mayinfer which combinations of program, program levels, and even specificsegments of programs are most effective at assisting users. Thisanalysis may be done across a variety of dimensions. For example,different groups may be assessed by age, location, physical activitylevels, online activity, and many other factors.

For each of the various permutations of characteristic combinations, andother factors, the server system 310 may identify examples from its dataset that represent the same or similar combination of aspects. Then,with information about how devices or users have progressed withdifferent combinations of programs and levels active, the server system310 can assign likelihood scores indicating, for example, a probabilitythat each level of each program is to achieve a desired result, such asreduction of a negative outcome. The same analysis may be done todetermine scores for different combinations of programs and levels.These likelihood scores may be used directly by the program statesetting module 453. As another example, the likelihood scores may beassessed by the server system 310 to define rules that the program statesetting module 453 applies to automatically initiate changes in programstates.

In some implementations, the server system 310 may determine thespecific set of factors experienced by an individual at a given time,and identify examples of users having the same or similar factors, basedon databases of information maintained by the server system 310. Theserver system 310 may thus define, at various times, custom data setsfrom historical information about other users to determine a userspecific likelihood of desired outcomes for the user's currentsituation. These user-specific measures may be used by the program statesetting module 453, along with user-specific risk levels discussedabove, to adjust program states.

FIG. 6B shows another example 650 of automatic transitions betweenprogram states. By contrast with FIG. 6A, a predetermined set ofconnections is used between the programs. For example, program 1 has itsstate adjusted based on the output or state of program 4, but thedecision does not take into account the output of or state of program 2and program 3.

In addition, rather than including a centralized program state settingmodule 453, the rules of each program set the state of the program oranother program. For example, program 4 may include rules that adjustthe level of program for based on the input data to the program, whichmay indicate historical information about the user. Program 4 may alsoprovide a signal to program 1 indicating that a level of program 1should be changed. Alternatively, the level of program 4, or outputs tobe provided to the user from program 4, maybe provided as input toprogram 1 and used to adjust program 1 according to program 1's ownrules.

FIG. 7 is a diagram that illustrates examples of using activityinformation of the device or user. The server system 310 can storeinformation about various activities or actions, as illustrated bydatabase 710. As examples, actions can be indicated by information suchas a name, category or type, etc. Scores indicating the severity of theeffect, e.g., an average measure for those who do experience the effect,can also be indicated

The server system 310 can also store a mapping 720 indicatingrelationships between the effects of actions and different programs. Forexample, for a given action effect, scores may be provided indicating adegree of relevance to different programs. The score may represent arelevance of the program or a degree to which the program can addressthe effect. The scores in the mapping 720 do not need to be based onintuitive relationships, however, and may be determined through analysisof activity data tracked over time or through historical outcomes ofusers of the system. In particular, the scores can be determined fordifferent outcomes based on the occurrence of certain digital statemarkers, or for certain progressions or transitions between digitalstate markers over time. The scores may be manually defined in someinstances. In another example, programs for devices such as sensor datahave a score in the mapping that relate to battery life and pullingsensor data, and a low score for relevance to results of pulling thesensor data. Different programs can have many effects: a high level ofsensor monitoring can decrease battery life and available computingresources such as CPU or RAM.

The server system 310 also stores activity records 730 for each deviceor user. These records 730 can indicate current actions or a currentstate of the device or the user as well as historical information aboutactions of a device or user. The records 730 can indicate a variety ofmetadata or contextual information about how and when the actions wereassigned. In addition, actual effects or conditions that the device oruser experienced can be specified, as reported by the user directlythrough a mobile device, or through sensor data.

Using the device or user's action records 730, and the indications ofwhich actions were taken when, the server system 310 can look up in theaction database 710 which action effects are likely for the device oruser. In addition, the server system 310 can use the mapping 720 todetermine which programs (and potentially specific levels and segmentsof the programs) are most relevant to the likely effects. Given a likelyeffect predicted using the action records, or from actual effectsindicated by the action records, the server system 310 identifies therelevance scores for one or more programs. The server system 310 maythen combine the scores across different effects.

For example, all effects of current actions of a device or user andformer actions of a device or user can be determined. The scores in themapping 720 are determined for each of these effects. Then, for eachprogram, the relevance scores may be combined by weighting the values.Scores corresponding to likely effects may be weighted or discountedaccording to the likelihood from the action database 710. Similarly,scores for effects estimated from actions that are not current may bediscounted in some instances based on how long ago the correspondingaction was administered. Similarly, scores can be weighted by thetypical intensity of the effect. By contrast, for actual effectsexperienced by the device or user, the scores from the mapping 720 maynot be discounted, or may be increased. The weighted scores for aprogram may then be added or otherwise combined to give an overallmeasure of the likely relevance of the program given the current stateor actions and history. The combined scores for the program are thencompared to a threshold, and if the score satisfies the threshold, thecorresponding program may be activated or have its level increased. Thesame technique can be used for each of the programs available.

With these techniques, the system 310 can activate digital operatingprograms to address the actual effects experienced and/or sensed by adevice or user. The system 310 can also activate programs to addresspotential effects that may be likely but may not be reported, or may bedeveloped in the future. The server system 310 can generate controlinstructions 740 that specify specific programs to activate, deactivate,or change in level based on the analysis of the device's or user'sactivity records 730, and based on the other information discussedabove.

In some implementations, management plans can be provided to manage theoperation of a device to improve a user's behavior or health. Forexample, a server-based management system can use various programs tocustomize instructions to a mobile device of a user, such as a user'scellular phone or smart watch. The management plans can support variousaspects of a user's wellbeing, such as physical, emotional, social, andmental health. Just as different programs may focus on different aspectsof a devices operation (e.g., different programs for battery life, wearand component lifespan, thermal management, etc.), different programsmay monitor and provide interventions to improve different aspects of auser's health (e.g., different programs for nutrition, exercise, sleep,smoking cessation, etc.). In some instances, individual programs orcombinations of programs may be specialized for treatment or managementof specific health conditions, especially long-lasting or chronicconditions such as diabetes, cancer and cancer survivorship, heartdisease, arthritis, etc. The systems and methods of the presentapplication can optionally perform or be combined with the techniquesdisclosed in U.S. patent application Ser. No. 15/803,556, filed on Nov.3, 2017, which is incorporated by reference herein.

Some management plans for a user can assist a user to reach a goal suchas losing weight, increasing sleep quality, creating a healthy habit,and so on. These user behaviors and outcomes that are affected are notmerely behaviors interacting with devices, but often represent changesin real-world aspects of the user's life outside of human-deviceinteraction. The management plan for a particular device and user, andthe timing for delivery of instructions causing presentation ofdifferent content by the particular device to the user, are dynamicallyselected and updated using a series of different digital operatingprograms that can address different aspects of a user's wellbeing. Asdiscussed below, each program can have separate sets of content, rules,assessments, and interventions for providing a customized, adaptiveexperience for a user. The system can operate in an “always-on” manner,frequently or continually assessing a data stream indicating the user'scurrent status and context, and providing targeted interactions that arerelevant to the user's current or estimated future needs.

Thus, the management plans can be applied to manage the operation ofdevices, whether a user's personal mobile device(s) or a medical device,to provide digital therapeutics or other therapy to treat healthconditions or improve health. In use, the server system (e.g., serversystem 310 of FIG. 3) may identify and provide management plans thatdecrease unwanted risks and increase likelihoods of improving a desiredarea of the user's life. For example, the predetermined set of programsthat the server system 310 uses to generate a management plan can bedigital therapeutics programs. The programs and levels set can specifyareas of the user's wellbeing to support, e.g., with different programsselected from among programs for nutrition, sleep, exercise, weightmanagement, alcohol use, smoking cessation, anxiety management,depression management, sexual health, pain management, fatiguemanagement, psychosocial health, adherence to physician instructions ortreatment regimens, etc. The combination of the different programs, atthe user-specific levels set and varied as discussed above, can instructthe user's device to provide a variety of interactions and interventionsthat can help change a user's behavior and actual health outcomes. Theseinteractions can include, for example, suggestions for behavior change,alerts, reminders, games, educational information, surveys, media,social media interactions, real-time communication, and so on.

As a result, a management plan can be used to provide precision medicineto different people. The specific interactions for any given person canbe custom selected based on the device-specific or user-specific set ofprogram states indicted by the management plan. Similarly, the timingand content of interactions over the course of a precision medicinetherapy are customized using the periodic or ongoing data stream thatthe server system 310 receives which indicates the behavior, physicalcharacteristics, or other attributes or actions of the user. Forexample, even when a certain type of interaction, such as a reminder orinstruction to a user, is determined to be appropriate for a user, thetime, location, and form of the presentation (e.g., text, image, video,audio, interactive game, real-time call to another person, etc.) may beinitiated by the server system 310 just in time, as triggered by rulesor other logic of the digital therapeutics programs.

The precision medicine provided by the system to an individual can bebased on data from genomics, pharmacogenomics, proteomics, metabolomics,multi-omics, and other analysis techniques for an individual. Thechanges to device operation that the management plan causes can beconfigured to cause interactions with the user that address any of avariety of health conditions, both physical and mental, such asdiabetes, cancer, heart disease, kidney disease, anxiety, depression,etc.

Initially, the management plan for a user can be set using a portal suchas the portal 332 shown in FIG. 3E, and can be provided to anadministrator, and in some implementations, to the user or to anotheruser, such as a family member of the user, a friend of the user, and/ora service provider for the user.

In some implementations, all of the programs used to define a managementplan or care plan for a user can be used to determine output of a singleapplication on a single device, such as a user's phone. The programs andlevels (e.g., program states) are adjusted dynamically, according to acombination of factors such as user inputs, user actions, and sensordata as discussed above. The level of a program may affect, for example,how frequently the program communicates with the user, how ambitious ordifficult the goals and activities of the program are, how closely aperson's behavior is monitored, and so on. Different levels maycorrespond to different levels of user need. For example, a motivationprogram may have a first level that helps a user monitor and understandtheir habits when they are trying to maintain a habit. A second levelmay provide more aggressive goal setting and motivation, with morefrequent interactions to help the user develop a new habit. For example,the program may interact with devices for friends and family members tomore actively assist in helping a user meet their goals. In someimplementations, different levels of a program, may respond to differentgoals or conditions.

In many cases, simple recommendations or questions by the server system310 can be targeted directly to alleviating symptoms or risks associatedwith the user's current status. For example, when the server system 310detects data indicative of anxiety or low energy, the server system 310can provide an indication of the current weather and encouragement tovisit a nearby state park. The rules of one or more programs may takeinto account that sunshine and physical activity are likely to addresssymptoms of low energy. In fact, through analysis of symptom levels andactivities of other users at a similar state, the server system 310 canset or adjust rules specifying when this suggestion should be provided.

In addition, the system may obtain information in ways that do notrequire explicit entry of an answer to a question. For example, if thesystem provides a content for user, the system can assess whether theuser completes viewing the content, which areas the user spends the mosttime viewing, which links are selected, and so on. These can providevaluable information that indicates a user's level of engagement, mood,etc. Similarly, a user's performance in a game may indicate the user'sreaction time, language capabilities, mood, and so on.

The server system 310 may cause many different types of interactions anddigital therapeutics interventions to be provided to a user. Forexample, the system 310 can cause a device to perform any of thefollowing actions: prompt a user to provide a journal entry or to view aprior journal entry; record a measurement from one or more devices;provide a survey or question for a user to answer; provide content for auser to read or view; initiate a challenge for a user with a definedtime frame (e.g., a daily, weekly, or monthly challenge goal orcompetition); prompt a user to set, adjust, or view a personal goal;communicate with family, friends, or others regarding a user's goals orstatus; and initiate a call, message exchange, or real-time text chatwith a service provider. The actions taken by the system can provideeducational information and activities to improve a user's knowledge.The actions can provide motivation or encouragement to increase positivebehaviors or decrease negative ones, for example, prior to a userperforming a desired action. The actions of the system can reinforcepositive behaviors, for example, by providing positive messages,rewards, or activities after a desired behavior is performed. These andother actions of the system can be performed as directed by the digitaloperating programs that are active for the user, with each activeprogram potentially providing different interactions at different times.The interventions of the programs can be tailored to achieve any ofvarious results, including adherence to a regimen, behavior change, usereducation, and so on.

For example, for a user who is trying to achieve a goal of increasingphysical activity, location data indicating that the user is near anexercise class may trigger the system to provide, at that time, helpfulinformation, such as a reminder of the user's current goal, praise forrecent progress, or information about the class.

In some implementations, the management plan may include informationabout friends and family members of the user, and phone numbers, emailaddresses, device identifiers, or other information that enables theserver system 310 to interact with them. For example, the server system310 may communicate with a family member's device and friend's device,as just a few examples. Just as the server system 310 can provideoccasional surveys or interactive activities to the user, server system310 can also generate customized surveys and activities for friends andfamily of the user. These interactions can provide confirmation of datathat the user provided, and/or different types of data, such asinformation that the user may not personally be aware of. The system caninitiate communication with computing systems for the user's physician,insurance company, social media network, calendar provider, or otherservice also.

Each of multiple devices can run an interactive application thatreceives and carries out instructions from the server system 310. All ofthe device may receive surveys, educational information, and otherinteractive activities. For example, when information is needed, or whencompliance with a goal needs to be verified, a friend or family membermay be asked in addition to or instead of the user. In someimplementations, friends or family members may remotely view someinformation about the user, including goals, progress toward goals,status information, and an indication of which programs are active. Thisinformation may be subject to consent of the user. A caregiver may beprovided this same information as well as indications of levels ofprograms, recent interactions, and other data gathered by the system.The caregiver's interface through the application may further allow thecaregiver to set goals or change the states of programs, while theinterface for friends and family may not allow these changes.

In some implementations, relationships between programs may bepredetermined, e.g., manually defined or explicitly set through rules.This is not required, however. Indeed, the analysis of the combinedprogram states in a care plan and/or the outputs of the programs to auser can be effective to adjust program states even with no specificknowledge of interconnections between the subject matter of differentapplications/programs. The server system 310 can infer relationshipsthrough commonalities in program specifications and outputs of theprograms. That is, the patterns of outputs of a program and requests forinputs by the program can show over time in the manner in which theprogram relates to other programs. The server system 310 may analyzethese relationships, for example, comparing which segments of programsand which levels of programs relate to specific levels and segment ofother programs. The system 310 can also apply machine learningtechniques, such as reinforcement learning, to patterns of care plancreation, level transitions over time, and other factors. With asufficient data set generated from the progress of many users, theserver system 310 can identify typical patterns in user status profiles,with the corresponding combinations of programs and levels. As thesystem 110 assesses program state transitions over time, the serversystem 310 is able to identify combinations of programs and levels thatproduce the best results for users with specific backgrounds. Forexample, the server system 110 may identify, for each of multipledifferent cluster of symptoms, which combinations of programs, and atwhat levels, are most effective at reducing the symptoms over a timeperiod, e.g., a week, a month, a year, etc.

For example, the server system 310 may evaluate user status profiles andcorresponding program states over a period of time, and determine thatusers that have three particular programs active simultaneously have thehighest likelihood of achieving a decrease in the level of a fourthprogram. From this determination, the server system 310 may generaterules that activate the three programs together if the fourth program isactive. More complex conditions may be set if desired. For example,rather than activate all three programs, the change may be limited toactivating only one of the programs if two of the three are alreadyactive and the fourth is also active. As another example, all threeprograms in the group may be activated if the level of the fourthprogram is above a threshold, but not if the level of the fourth programis below the threshold. Of course, the behavior learned of the serversystem 310 does not need to be expressed in explicit rules and can, invarious implementations, be incrementally learned as part of a trainingstate of a classifier, neural network, or other machine learning model.

In general, the server system 310 can use a variety of interactions,measurements, testing, biomarkers (e.g., physiological readings, bloodtest results, genetic information, etc.) and stated or observedbehaviors of a user to set program states. The server system 310 can usethese same factors to select which digital therapeutic interventions toprovide to an individual user, and when and how to provide them. Inaddition, the server system 310 can identify digital biomarkers and usethem as indicators for selecting certain digital therapeutics. Certaincombinations of data about a patient's activities and lifestyle canindicate health status and health risks of a user, just as the user'sblood chemistry, genetic profile, and other observable physical traitsmay indicate health status and health risks. Similarly, data that theserver system 110 collects about a user's activities and preferences, incombination with information about physical traits, may serve as digitalbiomarkers that provide more accurate predictive information than thephysical traits alone.

As an example, the server system 310 may define a particular biomarkerto represent a patient having a certain gene or combination of genes,with a sedentary activity profile, and certain diet characteristics.This combination of factors may be known to provide increased risk forcancer or other health conditions. When a user is determined to havethis combination of factors, which may be determined from digitalrecords of sensor data, behavioral data, device interaction patterns,and so on, the server system 310 may input to the various digitaltherapeutics programs that the digital biomarker is present. The variousprograms may then respond with interventions tailored to alter thebehaviors that cause the digital biomarker to be present, or toencourage other behaviors that decrease the risks associated with thedigital biomarker.

The server system 310 may evaluate various data sets to determine whichcombinations of data about a user constitute a digital biomarker that ispredictive of health outcomes. For example, the server system 310 mayaccess data sets for longitudinal studies that show data collectionsdescribing many individuals, their characteristics, and their healthoutcomes over time. From analysis of this data, the server system 310can select combinations of characteristics and actions that satisfy aminimum threshold of relevance to different health outcomes, and definethe identified combinations of data factors as digital biomarkers.

In some implementations, digital biomarkers can be defined in terms ofdigital therapeutics programs and their states. For example, by analysisof users and their health outcomes, the server system 310 may identifycombinations of programs and levels that, when active may representcertain risks, when taken alone or in combination with physical healthtraits. Accordingly, the server system 310 may define these combinationsof programs and states as biomarkers that may be used to alter thecombination of states, for example, to add a particular additionalprogram to reduce the risk of a future complication. Digital biomarkerscan be defined in terms of combinations of programs and program states,together with different combinations of physical, genetic, behavioral,or other traits. For example, a digital biomarker may represent aparticular combination of program states in a management plan combinedwith certain physical traits and/or certain behavioral events orpatterns detected over the course of the server system 310 administeringthe management plan.

An assessment module can determine the unique risk profile of a usergiven the user's current profile and historical data. Cancer patientsand cancer survivors often have drastically different risk profilescompared to others. From one cancer patient or survivor to the next, therisks maybe similarly diverse. As an example, a patient that hasundergone radiation therapy may have a higher risk of incidence of latercancers of other types. As another example, a breast cancer survivor mayhave a much higher risk of adverse health effects due to alcoholconsumption then a person that has not had breast cancer. A person'slifestyle, exposure to environmental factors, genetic profile, familymedical history, and many other factors result in unique risk levels forindividual patients. Because the server system 310 collects and storesinformation for these factors, the server system 310 can calculate theindividualized risks based on the data set compiled for the user. To aidin generating these risk levels, the server system 310 may store andaccess clinical data sets representing outcomes and statisticsrepresenting many different groups of people. From clinical data andstatistical analysis of the data sets, the server system 310 candetermine a baseline risk level as well as a large set of factors thatincrease and decrease risk. The server system 310 identifies which ofthe many factors are applicable given a user's current profile andhistorical data and adjusts the baseline risk accordingly. In thismanner, risk levels can be generated for many different conditions,e.g., pain, depression, recurrence of cancer, reduced sensory ability,etc.

The assessment module can determine an individual's risk with respect toa set of potential outcomes and update those as the person's profilechanges. Similarly, as the server system 310 receives new clinical dataor as user outcome patterns are observed for the users of the system,the risk levels are also changed. The risk levels determined by theassessment module may be used in a number of ways. In some instances, aperson may be informed of their risk level and how it compares to othercancer patients or survivors, or to other people in general. As anotherexample, the risk levels may be provided as input to the rules ofvarious programs, provided as input to the rules of various programs,which may condition they performance of certain system actions on risklevels in certain ranges. For example, a risk level for reduced mobilitymay be calculated and provided as an input to a program for physicalexercise, which may increase or decrease its level based on that risklevel. Risk levels for a patient may trigger the activation of a programor increase in the level of a program, e.g., if the user's risk ofexperiencing an associated condition is above a threshold. Similarly,risk levels can trigger the deactivation of a program or decrease in thelevel of a program, e.g., if the user's risk is below a threshold.

As discussed above, a program state transition module may governtransitions in program state for a user's management or care plan. Asdiscussed above, changes may be made due to the interaction andexecution of rules for different programs. In addition, or as analternative, a module of the system 110 may be used to manage thesechanges. This program state transition module may initiate changes inprograms based on analysis of a user status profile and current andprevious program states for the user. The program state transitionmodule 253 may limit or normalize changes indicated by program rules.For example, the program state transition module 253 may enforce certainrestrictions, such as limiting a frequency of level changes that areallowed over a period of time. An advantage of using the program statetransition module to initiate program state transitions is the abilityto apply analysis for the care plan as a whole, rather than just as therules of a single program within a care plan. As a result, the programstate transition module can enforce various aspects desired for careplans, whether determined using manually set rules, administratorpreferences, relationships learned through analysis of stored data, andso on.

The program state setting module 453 also receives data indicating thecurrent level and segment that is active for each program, and this dataalso is used by the module 453 to update the levels of the programs. Asshown in FIG. 4A, the program state setting module 453 may useinformation from any or all of the programs to set the state of any orall of the other programs. From data indicating the progress of varioususers over the course of using a care plan, the program state settingmodule may infer which combinations of program, program levels, and evenspecific segments of programs are most effective at assisting users.This analysis may be done across a variety of dimensions. For example,different groups may be assessed by symptoms experienced, intensity ofsymptoms, type of cancer experienced, stage of cancer or length of timein remission, types of treatments used, types of medications currentlyadministered or previously administered, age, location, nutritionchoices, physical activity levels, and many other factors.

For example, the server system 310 may determine that for a breastcancer survivor currently experiencing fatigue one year into remission,a certain set of programs and levels provides a high likelihood ofreducing the fatigue. Similarly, the analysis may show that a differentset of programs is determined to be effective for a patient currentlybeing treated with radiation for prostate cancer who has historicallyhad high activity levels but recently has reduced his exercise levels.

For each of the various permutations of symptom combinations, cancertypes, and other factors, the server system 310 may identify examplesfrom its data set that represent the same or similar combination ofaspects. Then, with information about how users have progressed withdifferent combinations of programs and levels active, the server system310 can assign likelihood scores indicating, for example, a probabilitythat each level of each program is to achieve a desired result, such asreduction of a symptom or other outcome. The same analysis may be doneto determine scores for different combinations of programs and levels.These likelihood scores may be used directly by the program statesetting module 453. As another example, the likelihood scores may beassessed by the server system 310 to define rules that the program statesetting module 453 applies to automatically initiate changes in programstates.

In some implementations, the server system 110 may determine thespecific set of factors experienced by an individual at a given time,and identify examples of users having the same or similar factors, basedon databases of information maintained by the server system 110. Theserver system 110 may thus define, at various times, custom data setsfrom historical information about other users to determine a userspecific likelihood of desired outcomes for the user's currentsituation. These user-specific measures may be used by the program statesetting module 253, along with user-specific risk levels discussedabove, to adjust program states.

In addition to or instead of using the centralized program state settingmodule 453, the rules of each program set the state of the program oranother program. For example, program 4 may include rules that adjustthe level of program for based on the input data to the program, whichmay indicate historical information about the user. Program 4 may alsoprovide a signal to program 1 indicating that a level of program 1should be changed. Alternatively, the level of program 4, or outputs tobe provided to the user from program 4, maybe provided as input toprogram 1 and used to adjust program 1 according to program 1's ownrules.

FIG. 8 illustrates additional examples of data that can be used byvarious management programs, such as management programs used to changethe operation of a device to support the health and wellbeing of a userof the device.

As illustrated, the server system 310 may receive various types of inputdata 810 such as survey responses, journal entries, interaction data,behavioral trends, location data, calendar data, social network data,electronic health records, prescription/medication data, genetic data,and other information about a user. Other input data 810 may representgeneral information, which may not be specific to a user can includeweather data, points of interest data, environmental data, and so on.The types of data about a user that may be tracked include caloriesconsumed, steps taken, distance walked, floors walked, elevation,minutes sedentary, minutes lightly active, minutes moderately active,minutes highly active, calories used during physical activity, waterconsumption, weight, body fat, heart rate, blood pressure, respirationrate, sleep times and amounts, glucose levels, wearable device data,medical device data, and so on.

A number of digital operating programs 820 are shown as examples. Theseprograms may each have their own sets of content and rules that governhow they interact with users. Each program may be activated anddeactivated separately to support a different aspect of a user'sactivities. Nevertheless, as discussed above, the levels or states ofthe programs may be adjusted based on the outputs and states of otherprograms to respond to a user's situation and to proactively providesupport for expected challenges. In the illustration, separate programsare shown for goal setting, habit maintenance, and schedule management.Each of these programs may use any and all of the inputs to the serversystem 310 to determine how to interact with a user. The specific subsetof programs that is active in a user's management plan at a given time,with the active segments and levels of those programs, is used toprocess the input data 810 and provide a customized management plan andinteractive user interfaces 830.

As discussed above, the states of different programs can be dynamicallyadjusted, based on current information about a user, historicalinformation about the user, and based on the states of other programs.In addition, the types of interconnections between programs, e.g., therules that define transitions between program states can alsodynamically updated based on various factors. For example, the systemcan use information about the progress of users over time can be used toidentify conditions or triggers that should cause state transitions.Groups of users that have certain commonalities can be identified andtheir progress assessed to determine these conditions and triggers, andwhich actions to perform, e.g., which programs to activate ordeactivate, and which levels (e.g., program states) are most effective.In addition to or instead of using data about users of the system,population-level data can be used in a similar manner to determine whichcombinations of programs and interventions are appropriate for differentusers. The population-level data may represent information about apopulation of a city, county, state or province, country, continent, orthe world. Combining information from the data sets of users of thesystem with population-level data can provide increased accuracy ofpredictions, better enabling the system to identify predictedinteractions and interventions that will address the user's current orexpected needs.

Embodiments of the invention and all of the functional operationsdescribed in this specification may be implemented in digital electroniccircuitry, or in computer software, firmware, or hardware, including thestructures disclosed in this specification and their structuralequivalents, or in combinations of one or more of them. Embodiments ofthe invention may be implemented as one or more computer programproducts, i.e., one or more modules of computer program instructionsencoded on a computer-readable medium for execution by, or to controlthe operation of, data processing apparatus. The computer readablemedium may be a non-transitory computer readable storage medium, adevice-readable storage device, a device-readable storage substrate, amemory device, a composition of matter effecting a device-readablepropagated signal, or a combination of one or more of them. The term“data processing apparatus” encompasses all apparatus, devices, anddevices for processing data, including by way of example a programmableprocessor, a computer, or multiple processors or computers. Theapparatus may include, in addition to hardware, code that creates anexecution environment for the computer program in question, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them. A propagated signal is an artificially generated signal, e.g.,a device-generated electrical, optical, or electromagnetic signal thatis generated to encode information for transmission to suitable receiverapparatus.

A computer program (also known as a program, software, softwareapplication, script, or code) may be written in any form of programminglanguage, including compiled or interpreted languages, and it may bedeployed in any form, including as a stand-alone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program may be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programmay be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification may beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows may also be performedby, and apparatus may also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Moreover, a computer may be embedded inanother device, e.g., a tablet computer, a mobile telephone, a personaldigital assistant (PDA), a mobile audio player, a Global PositioningSystem (GPS) receiver, to name just a few. Computer readable mediasuitable for storing computer program instructions and data include allforms of non-volatile memory, media, and memory devices, including byway of example semiconductor memory devices, e.g., EPROM, EEPROM, andflash memory devices; magnetic disks, e.g., internal hard disks orremovable disks; magneto optical disks; and CD ROM and DVD-ROM disks.The processor and the memory may be supplemented by, or incorporated in,special purpose logic circuitry.

To provide for interaction with a user, embodiments of the invention maybe implemented on a computer having a display device, e.g., a CRT(cathode ray tube) or LCD (liquid crystal display) monitor, fordisplaying information to the user and a keyboard and a pointing device,e.g., a mouse or a trackball, by which the user may provide input to thecomputer. Other kinds of devices may be used to provide for interactionwith a user as well; for example, feedback provided to the user may beany form of sensory feedback, e.g., visual feedback, auditory feedback,or tactile feedback; and input from the user may be received in anyform, including acoustic, speech, or tactile input.

Embodiments of the invention may be implemented in a computing systemthat includes a back end component, e.g., as a data server, or thatincludes a middleware component, e.g., an application server, or thatincludes a front end component, e.g., a client computer having agraphical user interface or a Web browser through which a user mayinteract with an implementation of the invention, or any combination ofone or more such back end, middleware, or front end components. Thecomponents of the system may be interconnected by any form or medium ofdigital data communication, e.g., a communication network. Examples ofcommunication networks include a local area network (“LAN”) and a widearea network (“WAN”), e.g., the Internet.

The computing system may include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specifics, these should not beconstrued as limitations on the scope of the invention or of what may beclaimed, but rather as descriptions of features specific to particularembodiments of the invention. Certain features that are described inthis specification in the context of separate embodiments may also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment mayalso be implemented in multiple embodiments separately or in anysuitable subcombination. Moreover, although features may be describedabove as acting in certain combinations and even initially claimed assuch, one or more features from a claimed combination may in some casesbe excised from the combination, and the claimed combination may bedirected to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems maygenerally be integrated together in a single software product orpackaged into multiple software products.

In each instance where an HTML file is mentioned, other file types orformats may be substituted. For instance, an HTML file may be replacedby an XML, JSON, plain text, or other types of files. Moreover, where atable or hash table is mentioned, other data structures (such asspreadsheets, relational databases, or structured files) may be used.

Thus, particular embodiments of the invention have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims may be performed in a different orderand still achieve desirable results.

What is claimed is:
 1. A method comprising: storing, by a server system,data indicating a plan associated with a user of a device that isconfigured to communicate with the server system over a network, theplan indicating a user-specific set of program states for programs in apredetermined plurality of programs, each of the programs havingmultiple predetermined program states that can be assigned for theprograms, the predetermined plurality of programs comprising at least afirst program and a second program that is different from the firstprogram, wherein the first program and the second program are bothconfigured to cause interactions with the user; altering, by the serversystem, the plan associated with the user by assigning, in the alteredplan associated with the user, an altered user-specific set of programstates for the plurality of programs comprising program states selectedfrom among the predetermined program states for the respective programs,wherein the altered plan assigns a first program state for the firstprogram from among the multiple predetermined program states for thefirst program, wherein the first program state is determined based on asecond program state that is associated with the user and is one of themultiple predetermined program states for the second program;generating, by the server system, an instruction that alters operationof the device based on the altered user-specific set of program statesassigned in the altered plan, wherein the instruction is determined byevaluating data collected about the user or the device, wherein: theevaluation is performed using rules for one or more of the programs thatare active according to the altered user-specific set of program statesassigned in the altered plan; the first program has different sets ofrules that correspond to different program states of the first program;and the evaluation includes evaluating data collected for the user usingthe set of rules that corresponds to the assigned first program statefor the first program; and transmitting, by the server system and to thedevice over the network, the generated instruction determined accordingto the user-specific set of program states assigned in the altered plan,wherein the instruction is configured to cause the device to perform oneor more operations associated with the altered plan.
 2. The method ofclaim 1, further comprising: defining, by the server system, a marker toinclude a set of factors that includes a particular combination ofprograms in the plurality of programs being concurrently active and aparticular set of program states being set for the active programs;identifying, by the server system, a group of users that have previouslyexhibited the marker by having the particular combination of programsbeing concurrently active with the particular set of program statesbeing set for the active programs; and identifying, by the serversystem, program state transitions that were made for the users in thegroup after the particular set of factors for the marker.
 3. The methodof claim 2, further comprising: determining, by the server system, thatthe user has exhibited the marker based on a user-specific set ofprogram states for the user setting the particular combination ofprograms to be concurrently active with the particular set of programstates being set for the active programs; updating, by the serversystem, the user-specific set of program states of the plurality ofprograms to assign updated program states for the user, wherein theserver system assigns the updated program states for the user inresponse to determining that the user has exhibited the marker and basedon the program state transitions made for the users in the group afterthe users in the group exhibited the marker; and distributing, by theserver system, content to the device that is selected for the useraccording to the altered user-specific set of program states.
 4. Themethod of claim 1, wherein the device is associated with a user, andwherein the predetermined plurality of programs comprises digitaltherapeutics programs, wherein altering the plan comprises alteringprogram states for the digital therapeutics programs to adjust precisionmedicine therapy for the user.
 5. The method of claim 1, wherein thesecond program state is the program state in the user-specific set ofprogram states for the user that corresponds to the second program. 6.The method of claim 1, wherein the second program state is associatedwith the user by being currently or previously assigned for the secondprogram in a plan associated with the user.
 7. The method of claim 1,comprising: providing, by the server system, content to the device overthe network, wherein the content is generated using the plurality ofprograms and the user-specific set of program states assigned in thealtered plan associated with the user.
 8. The method of claim 1, whereinthe instruction is configured to cause the device to record ameasurement from one or more devices that are in communication with thedevice through a wired or wireless data connection.
 9. The method ofclaim 1, wherein the instruction is determined by evaluating, for eachof the programs that is active according to the altered plan, a propersubset of rules for the program, the subset being associated with aparticular program state for the program indicated in the altered plan.10. The method of claim 1, wherein, for at least one of the programs inthe plurality of programs, the program states for the program specifydifferent levels of intensity or interaction for the program.
 11. Themethod of claim 1, wherein the instruction is configured to cause thedevice to provide an output at the device, wherein the output comprisespresentation of text, image, audio, or video by the device or anotherdevice in communication with the device.
 12. The method of claim 1,wherein altering the plan associated with the user comprises:automatically enabling one or more digital therapeutics programs for theuser such that the server system initiates use of the enabled digitaltherapeutics program to generate content customized for the user,communicate the customized content to the device, and instruct thedevice to present the customized content; or automatically disabling oneor more digital therapeutics programs for the user such that the serversystem discontinues use of the disabled digital therapeutics program togenerate customized content for the user.
 13. The method of claim 1,wherein, for at least one of the programs in the plurality of programs,the program states for the program correspond to different time periodsrepresenting sequential progression through a set of predeterminedelements of the program.
 14. The method of claim 1, wherein the firstprogram is a first digital therapeutics program configured to provideinteractions with the user through the device, and wherein the secondprogram is a second digital therapeutics program configured to provideinteractions with the user through the device.
 15. The method of claim1, wherein the first program and the second program are digitaltherapeutics programs, and wherein the digital therapeutics programs areconfigured to provide monitoring of the user and interactions with theuser for different areas of health of the user.
 16. The method of claim1, wherein the first program comprises a digital therapeutics programconfigured to provide interventions to support physical health of theuser, and wherein the second program comprises a digital therapeuticsprogram configured to provide interventions to support mental health ofthe user.
 17. The method of claim 1, comprising, in response to alteringthe plan associated with the user, setting the plurality of programs tointeract with the user at the respective program states indicated in thealtered user-specific set of program states.
 18. The method of claim 1,wherein the altered user-specific set of program states assigns aprogram state for the second program based on one or more program statesof the other programs in the plurality of programs.
 19. The method ofclaim 1, wherein the program states in the altered user-specific set ofprogram states for the user are determined based on data collected aboutthe user by one or more of the plurality of programs.
 20. The method ofclaim 1, wherein, for each of the predetermined plurality of programs,the predetermined program states for the program are different settingsthat can be assigned for the program to set a manner in which theprogram interacts with the user.
 21. A system comprising: one or morecomputers; and one or more non-transitory computer-readable mediastoring instructions that, when executed by one or more computers of aserver system, cause the server system to perform operations comprising:storing, by the server system, data indicating a plan associated with auser of a device that is configured to communicate with the serversystem over a network, the plan indicating a user-specific set ofprogram states for programs in a predetermined plurality of programs,each of the programs having multiple predetermined program states thatcan be assigned for the programs, the predetermined plurality ofprograms comprising at least a first program and a second program thatis different from the first program, wherein the first program and thesecond program are both configured to cause interactions with the user;altering, by the server system, the plan associated with the user byassigning, in the altered plan associated with the user, an altereduser-specific set of program states for the plurality of programscomprising program states selected from among the predetermined programstates for the respective programs, wherein the altered plan assigns afirst program state for the first program from among the multiplepredetermined program states for the first program, wherein the firstprogram state is determined based on a second program state that isassociated with the user and is one of the multiple predeterminedprogram states for the second program; generating, by the server system,an instruction that alters operation of the device based on the altereduser-specific set of program states assigned in the altered plan,wherein the instruction is determined by evaluating data collected aboutthe user or the device, wherein: the evaluation is performed using rulesfor one or more of the programs that are active according to the altereduser-specific set of program states assigned in the altered plan; thefirst program has different sets of rules that correspond to differentprogram states of the first program; and the evaluation includesevaluating data collected for the user using the set of rules thatcorresponds to the assigned first program state for the first program;and transmitting, by the server system and to the device over thenetwork, the generated instruction determined according to theuser-specific set of program states assigned in the altered plan,wherein the instruction is configured to cause the device to perform oneor more operations associated with the altered plan.
 22. The system ofclaim 21, wherein the one or more computers are configured to carry outand update plans for a plurality of different users, each of the plansindicating respective sets of program states for programs in theplurality of programs; wherein the server system is configured to: storedata indicating the plans and user-specific sets of program states foreach of the plurality of different users; alter the plans for each ofthe plurality of different users by changing the program statesspecified by the plans; generate customized instructions for devicesassociated with the different users according to the different sets ofprogram states indicated by the plans for the different users; and sendthe respective customized instructions to the devices over the network.23. The system of claim 21, wherein the server system is configured to:obtain sensor data from the device; and vary one or more program statesof the plan automatically based at least in part on the sensor datagenerated by the device.
 24. The system of claim 21, wherein the serversystem is configured to vary the program states specified by the planassociated with the user, wherein the program states are varied by theserver system according to data from: (i) at least one general datasource providing one or more of weather data, points of interest data,or environmental data; and (ii) at least one user-specific data sourcefrom the group consisting of data from a wireless wearable device of theuser, data from a medical device of the user, electronic health recordsfor the user, prescription data for the user, genetic data for the user,social media data for the user, and mobile device data from a mobiledevice associated with the user.
 25. The system of claim 21, wherein theserver system is configured to automatically vary the program stateseach of a plurality of different devices based on behavioral patterns ofrespective users of the plurality of different devices to instruct theplurality of different devices to initiate digital therapeuticsinterventions for the respective users of the plurality of differentdevices.
 26. One or more non-transitory computer-readable media storinginstructions that, when executed by one or more computers of a serversystem, cause the server system to perform operations comprising:storing, by the server system, data indicating a plan associated with auser of a device that is configured to communicate with the serversystem over a network, the plan indicating a user-specific set ofprogram states for programs in a predetermined plurality of programs,each of the programs having multiple predetermined program states, thatcan be assigned for the programs, the predetermined plurality ofprograms comprising at least a first program and a second program thatis different from the first program, wherein the first program and thesecond program are both configured to cause interactions with the user;altering, by the server system, the plan associated with the user byassigning, in the altered plan associated with the user, an altereduser-specific set of program states for the plurality of programscomprising program states selected from among the predetermined programstates for the respective programs, wherein the altered plan assigns afirst program state for the first program from among the multiplepredetermined program states for the first program, wherein the firstprogram state is determined based on a second program state that isassociated with the user and is one of the multiple predeterminedprogram states for the second program; generating, by the server system,an instruction that alters operation of the device based on the altereduser-specific set of program states assigned in the altered plan,wherein the instruction is determined by evaluating data collected aboutthe user or the device, wherein: the evaluation is performed using rulesfor one or more of the programs that are active according to the altereduser-specific set of program states assigned in the altered plan; thefirst program has different sets of rules that correspond to differentprogram states of the first program; and the evaluation includesevaluating data collected for the user using the set of rules thatcorresponds to the assigned first program state for the first program;and transmitting, by the server system and to the device over thenetwork, the generated instruction determined according to theuser-specific set of program states assigned in the altered plan,wherein the instruction is configured to cause the device to perform oneor more operations associated with the altered plan.